elasticsearch-dsl聚合-2

接续上篇，本篇介绍elasticsearch聚合查询，使用python库elasticsearch-dsl进行聚合查询操作。

条形图

聚合有一个令人激动的特性就是能够十分容易地将数据转换成图表和图形。

• • 创建直方图需要指定一个区间，如果我们要为售价创建一个直方图，可以将间隔设为 20,000。这样做将会在每个 $20,000 档创建一个新桶，然后文档会被分到对应的桶中。

     GET cars/transactions/_search
     {
       "size": ,
       "aggs": {
         "price": {
           "histogram": {
             "field": "price",
             "interval":
           },
           "aggs": {
             "revenue": {
               "sum": {
                 "field": "price"
               }
             }
           }
         }
       }
     }

     s = Search(index='cars')
     s.aggs.bucket("price", "histogram", field="price", interval=20000).metric("revenue", "sum", field="price")
     response = s.execute()

    图形化表示

  • 更强大的统计

     GET /cars/transactions/_search
     {
       "size" : ,
       "aggs": {
         "makes": {
           "terms": {
             "field": "make",
             "size":
           },
           "aggs": {
             "stats": {
               "extended_stats": {
                 "field": "price"
               }
             }
           }
         }
       }
     }

     s = Search(index='cars')
     s.aggs.bucket("makes", "terms", field="make", size=10).metric("stats", "extended_stats", field="price")
     response = s.execute()

  • 按时间统计（date_histogram），每月销售了多少台汽车？

     GET cars/transactions/_search
     {
       "size": ,
       "aggs": {
         "sales": {
           "date_histogram": {
             "field": "sold",
             "interval": "month",
             "format": "yyyy-MM-dd",
             "extended_bounds": {
               "min": "2014-01-01",
               "max": "2014-12-31"
             }
           }
         }
       }
     }

     s = Search(index='cars')
     s.aggs.bucket("sales", "date_histogram", field="sold", interval="month",
                       format="yyyy-MM-dd", extended_bounds={"min": "2014-01-01", "max": "2014-12-31"})
     response = s.execute()

  • 计算每个季度所有汽车品牌的销售总额以及每种汽车品牌的销售总额

     GET cars/transactions/_search
     {
       "size": ,
       "aggs": {
         "sales": {
           "date_histogram": {
             "field": "sold",
             "interval": "quarter",
             "format": "yyyy-MM-dd",
             "extended_bounds": {
               "min": "2014-01-01",
               "max": "2014-12-31"
             }
           },
           "aggs": {
             "per_make_sum": {
               "terms": {
                 "field": "make"
               },
               "aggs": {
                 "sum_price": {
                   "sum": {
                     "field": "price"
                   }
                 }
               }
             },
             "total_sum": {
               "sum": {
                 "field": "price"
               }
             }
           }
         }
       }
     }

     s = Search(index='cars')
     a1 = A("date_histogram", field="sold", interval="quarter", format="yyyy-MM-dd",
                extended_bounds={"min": "2014-01-01", "max": "2014-12-31"})
     a2 = A("terms", field="make")
     s.aggs.bucket("sales", a1).bucket("per_make_sum", a2).metric("sum_price", "sum", field="price")
     s.aggs["sales"].metric("total_sum", "sum", field="price")
     response = s.execute()

  • 限定范围的聚合，福特在售车有多少种颜色？

     GET cars/transactions/_search
     {
       "query": {
         "match": {
           "make": "ford"
         }
       },
       "aggs": {
         "colors": {
           "terms": {
             "field": "make"
           }
         }
       }
     }

     s = Search(index="cars").query("match", make="ford")
     s.aggs.bucket("colors", "terms", field="make")
     response = s.execute()

  • 全局桶（全局桶包含所有的文档，它无视查询的范围），比方说我们想知道福特汽车与所有汽车平均售价的比较

     GET cars/transactions/_search
     {
       "query": {
         "match": {
           "make": "ford"
         }
       },
       "aggs": {
         "single_avg_price": {
           "avg": {
             "field": "price"
           }
         },
         "all": {
           "global": {},         --global忽略过滤条件
           "aggs": {
             "avg_price": {
               "avg": {
                 "field": "price"
               }
             }
           }
         }
       }
     }

     s = Search(index="cars").query("match", make="ford")
     s.aggs.metric("single_avg_price", "avg", field="price")
     s.aggs.bucket("all", "global").metric("avg_price", "avg", field="price")
     response = s.execute()

  • 过滤，找到售价在 $10,000 美元之上的所有汽车同时也为这些车计算平均售价

     GET cars/transactions/_search
     {
       "query": {
         "constant_score": {
           "filter": {
             "range": {
               "price": {
                 "gte":
               }
             }
           }
         }
       },
       "aggs": {
         "single_avg_price": {
           "avg": {
             "field": "price"
           }
         }
       }
     }

     s = Search(index="cars").query("range", price={"gte": 10000})
     s.aggs.metric("single_avg_price", "avg", field="price")
     response = s.execute()

  • 过滤桶（一种特殊桶），搜索福特汽车在2014年上半年销售汽车的均价

     GET /cars/transactions/_search
     {
        "size" : ,
        "query":{
           "match": {
              "make": "ford"
           }
        },
        "aggs":{
           "recent_sales": {
              "filter": {
                 "range": {
                    "sold": {
                       "from": "2014-01-01",
                       "to": "2014-06-30"
                    }
                 }
              },
              "aggs": {
                 "average_price":{
                    "avg": {
                       "field": "price"
                    }
                 }
              }
           }
        }
     }

     s = Search(index="cars").query("match", make="ford")
     q = Q("range", sold={"from": "2014-01-01", "to": "2014-06-30"})
     s.aggs.bucket("recent_sales", "filter", q).metric("average_price", "avg", field="price")
     response = s.execute()

  • 后过滤器（post_filter），只过滤搜索结果，不过滤聚合结果，对聚合没有影响

     GET cars/transactions/_search
     {
    
       "query": {
         "match": {
           "make": "ford"
         }
       },
       "post_filter": {
         "term": {
           "color": "green"
         }
       },
       "aggs": {
         "all_colors": {
           "terms": {
             "field": "color"
           }
         }
       }
     }

     s = Search(index="cars").query("match", make="ford").post_filter("term", color="green")
     s.aggs.bucket("all_colors", "terms", field="color")
     response = s.execute()

内置排序

• _count：按文档数排序。对 terms 、 histogram 、 date_histogram 有效
• _term：按词项的字符串值的字母顺序排序。只在 terms 内使用
• _key：按每个桶的键值数值排序（理论上与 _term 类似）。 只在 histogram 和 date_histogram 内使用

• • 让我们做一个 terms 聚合但是按 doc_count 值的升序排序

     GET cars/transactions/_search
     {
       "size": ,
       "aggs": {
         "colors": {
           "terms": {
             "field": "color",
             "order": {
               "_count": "asc"
             }
           }
         }
       }
     }

     s = Search(index="cars")
     s.aggs.bucket("colors", "terms", field="color", order={"_count": "asc"})
     response = s.execute()

  • 按度量排序，按照汽车颜色分类，再按照汽车平均售价升序排列

     GET cars/transactions/_search
     {
       "size": ,
       "aggs": {
         "colors": {
           "terms": {
             "field": "color",
             "order": {
               "avg_price": "asc"
             }
           },
           "aggs": {
             "avg_price": {
               "avg": {
                 "field": "price"
               }
             }
           }
         }
       }
     }

     s = Search(index="cars")
     s.aggs.bucket("colors", "terms", field="color", order={"avg_price": "asc"}).metric("avg_price", "avg", field="price")
     response = s.execute()

  • 基于“深度”度量排序

我们可以定义更深的路径，将度量用尖括号（ > ）嵌套起来，像这样： my_bucket>another_bucket>metric 。

需要提醒的是嵌套路径上的每个桶都必须是 单值 的。 filter 桶生成 一个单值桶：所有与过滤条件匹配的文档都在桶中。 多值桶（如：terms ）动态生成许多桶，无法通过指定一个确定路径来识别。

目前，只有三个单值桶： filter 、 global 和 reverse_nested 。

• • 让我们快速用示例说明，创建一个汽车售价的直方图，但是按照红色和绿色（不包括蓝色）车各自的方差来排序

     GET /cars/transactions/_search
     {
         "size" : ,
         "aggs" : {
             "colors" : {
                 "histogram" : {
                   "field" : "price",
                   "interval": ,
                   "order": {
                     "red_green_cars>stats.variance" : "asc"
                   }
                 },
                 "aggs": {
                     "red_green_cars": {
                         "filter": { "terms": {"color": ["red", "green"]}},
                         "aggs": {
                             "stats": {"extended_stats": {"field" : "price"}}
                         }
                     }
                 }
             }
         }
     }

     s = Search(index="cars")
     a = A("histogram", field="price", interval=20000, order={"red_green_cars>stats.variance": "asc"})
     q = A("filter", filter={"terms": {"color": ["red", "green"]}})
     s.aggs.bucket("colors", a).bucket("red_green_cars", q).metric("stats", "extended_stats", field="price")
     response = s.execute()