Ignoring TF/IDF

Sometimes we just don’t care about TF/IDF. All we want to know is that a certain word appears in a field. Perhaps we are searching for a vacation home and we want to find houses that have as many of these features as possible:

  • WiFi
  • Garden
  • Pool

The vacation home documents look something like this:

{ "description": "A delightful four-bedroomed house with ... " }

We could use a simple match query:

GET /_search
{
"query": {
"match": {
"description": "wifi garden pool"
}
}
}

However, this isn’t really full-text search. In this case, TF/IDF just gets in the way. We don’t care whether wifi is a common term, or how often it appears in the document. All we care about is that it does appear. In fact, we just want to rank houses by the number of features they have—the more, the better. If a feature is present, it should score 1, and if it isn’t, 0.

constant_score Query

Enter the constant_score query. This query can wrap either a query or a filter, and assigns a score of1 to any documents that match, regardless of TF/IDF:

GET /_search
{
"query": {
"bool": {
"should": [
{ "constant_score": {
"query": { "match": { "description": "wifi" }}
}},
{ "constant_score": {
"query": { "match": { "description": "garden" }}
}},
{ "constant_score": {
"query": { "match": { "description": "pool" }}
}}
]
}
}
}

Perhaps not all features are equally important—some have more value to the user than others. If the most important feature is the pool, we could boost that clause to make it count for more:

GET /_search
{
"query": {
"bool": {
"should": [
{ "constant_score": {
"query": { "match": { "description": "wifi" }}
}},
{ "constant_score": {
"query": { "match": { "description": "garden" }}
}},
{ "constant_score": {
"boost": 2
"query": { "match": { "description": "pool" }}
}}
]
}
}
}

A matching pool clause would add a score of 2, while the other clauses would add a score of only 1 each.

The final score for each result is not simply the sum of the scores of all matching clauses. The coordination factor and query normalization factor are still taken into account.

We could improve our vacation home documents by adding a not_analyzed features field to our vacation homes:

{ "features": [ "wifi", "pool", "garden" ] } 这样改写有什么好处?省索引空间吗?

参考:https://www.elastic.co/guide/en/elasticsearch/guide/current/ignoring-tfidf.html#ignoring-tfidf

ES忽略TF-IDF评分——使用constant_score的更多相关文章

  1. Elasticsearch由浅入深(十)搜索引擎:相关度评分 TF&IDF算法、doc value正排索引、解密query、fetch phrase原理、Bouncing Results问题、基于scoll技术滚动搜索大量数据

    相关度评分 TF&IDF算法 Elasticsearch的相关度评分(relevance score)算法采用的是term frequency/inverse document frequen ...

  2. Elasticsearch学习之相关度评分TF&IDF

    relevance score算法,简单来说,就是计算出,一个索引中的文本,与搜索文本,他们之间的关联匹配程度 Elasticsearch使用的是 term frequency/inverse doc ...

  3. 25.TF&IDF算法以及向量空间模型算法

    主要知识点: boolean model IF/IDF vector space model     一.boolean model     在es做各种搜索进行打分排序时,会先用boolean mo ...

  4. TF/IDF计算方法

    FROM:http://blog.csdn.net/pennyliang/article/details/1231028 我们已经谈过了如何自动下载网页.如何建立索引.如何衡量网页的质量(Page R ...

  5. 信息检索中的TF/IDF概念与算法的解释

    https://blog.csdn.net/class_brick/article/details/79135909 概念 TF-IDF(term frequency–inverse document ...

  6. 55.TF/IDF算法

    主要知识点: TF/IDF算法介绍 查看es计算_source的过程及各词条的分数 查看一个document是如何被匹配到的         一.算法介绍 relevance score算法,简单来说 ...

  7. TF/IDF(term frequency/inverse document frequency)

    TF/IDF(term frequency/inverse document frequency) 的概念被公认为信息检索中最重要的发明. 一. TF/IDF描述单个term与特定document的相 ...

  8. 基于TF/IDF的聚类算法原理

        一.TF/IDF描述单个term与特定document的相关性TF(Term Frequency): 表示一个term与某个document的相关性. 公式为这个term在document中出 ...

  9. 使用solr的函数查询,并获取tf*idf值

    1. 使用函数df(field,keyword) 和idf(field,keyword). http://118.85.207.11:11100/solr/mobile/select?q={!func ...

随机推荐

  1. 应该知道的Linux技巧(转载)

    这篇文章来源于Quroa的一个问答<What are some time-saving tips that every Linux user should know?>—— Linux用户 ...

  2. RelativeLayout布局(仅在RelativeLayout中有效)

    在父亲布局的相对位置 android:layout_alignParentLeft="true"     //在布局左边 android:layout_alignParentRig ...

  3. scikit-learn(project中用的相对较多的模型介绍):2.3. Clustering(可用于特征的无监督降维)

    參考:http://scikit-learn.org/stable/modules/clustering.html 在实际项目中,我们真的非常少用到那些简单的模型,比方LR.kNN.NB等.尽管经典, ...

  4. java 中的CountDownLatch

    直接使用thread可以使用thread和wait notify 实现顺序执行 线程池中可以使用CountDownLatch 进行顺序执行 package com.test; import java. ...

  5. ZIP解压缩文件的工具类【支持多级文件夹|全】

    ZIP解压缩文件的工具类[支持多级文件夹|全] 作者:Vashon 网上有非常多的加压缩演示样例代码.可是都仅仅是支持一级文件夹的操作.假设存在多级文件夹的话就不行了. 本解压缩工具类经过多次检查及重 ...

  6. Windows系统的Jenkins持续集成环境

    Windows系统的Jenkins持续集成环境 如题:本文将介绍如何在Windows环境下运用Jenkins部署持续集成环境.之所以写本文,是因为在最近工作当中,学习使用Jenkins时,确实遇到了一 ...

  7. Tomcat 7.0 servlet @WebServlet

    在使用tomcat7.0+eclipse j2ee时,新建Dynamic Web Project时, 会让选择是否生成web.xml.无论选择与否,此时新建一个servlet, 可以不在web.xml ...

  8. 【puppeteer+Node.js】学习

    总结了一下有关puppeteer的学习的网站,以后还会继续更新 puppeteer 介绍 Puppeteer是一个通过DevTools Protocol控制headless chromium的高级no ...

  9. PowerBuilder -- 其他

    判断某键是否被按下 KeyDown ( keycode ) 继承问题 如果是 uf_1是函数呢   你在父类UO_1的uf_1里面 写了代码,只要在子类UO_2的uf_1写了代码,默认是覆盖(over ...

  10. Ubuntu64位安装Adobe Reader 9.5.5

    Aodbe Reader在Linux下的效果比Foxit Reader(福昕阅读器)要好一些,尤其对于中文文档而言.本文介绍Adobe Reader在Ubuntu下的安装,文章<Ubuntu12 ...