search（13）- elastic4s-histograms：聚合直方图
在聚合的分组统计中我们会面临两种分组元素类型：连续型如时间，自然数等、离散型如地点、产品等。离散型数据本身就代表不同的组别，但连续型数据则需要手工按等长间隔进行切分了。下面是一个按价钱段聚合的例子：
POST /cartxns/_search
{
"size" : ,
"aggs": {
"sales_per_pricerange": {
"histogram": {
"field": "price",
"interval":
},
"aggs": {
"total sales": {
"sum": {
"field": "price"
}
}
}
}
}
}
}
在上面这个例子中我们把价钱按20000进行分段。得出0-19999,20000-39999,40000-59999 ... 价格段的度量：
"aggregations" : {
"sales_per_pricerange" : {
"buckets" : [
{
"key" : 0.0,
"doc_count" : ,
"total sales" : {
"value" : 37000.0
}
},
{
"key" : 20000.0,
"doc_count" : ,
"total sales" : {
"value" : 95000.0
}
},
{
"key" : 40000.0,
"doc_count" : ,
"total sales" : {
"value" : 0.0
}
},
{
"key" : 60000.0,
"doc_count" : ,
"total sales" : {
"value" : 0.0
}
},
{
"key" : 80000.0,
"doc_count" : ,
"total sales" : {
"value" : 80000.0
}
}
]
}
}
在elastic4s中是这样表达的：
val aggHist = search("cartxns").aggregations(
histogramAggregation("sales_per_price")
.field("price")
.interval().subAggregations(
sumAggregation("total_sales").field("price")
)
)
println(aggHist.show)
val histResult = client.execute(aggHist).await
if (histResult.isSuccess)
histResult.result.aggregations.histogram("sales_per_price").buckets
.foreach(hb => println(s"${hb.key},${hb.docCount}:${hb.sum("total_sales").value}"))
else println(s"error: ${histResult.error.reason}")
....
POST:/cartxns/_search?
StringEntity({"aggs":{"sales_per_price":{"histogram":{"interval":20000.0,"field":"price"},"aggs":{"total_sales":{"sum":{"field":"price"}}}}}},Some(application/json))
0.0,:37000.0
20000.0,:95000.0
40000.0,:0.0
60000.0,:0.0
80000.0,:80000.0
下面这个按车款分组统计的就是一个离散元素的聚合统计了：
POST /cartxns/_search
{
"size" : ,
"aggs": {
"avage price per model" : {
"terms": {"field" : "make.keyword"},
"aggs": {
"average price": {
"avg": {"field": "price"}
},
"max price" : {
"max": {
"field": "price"
}
},
"min price" : {
"min": {
"field": "price"
}
} }
}
}
}
我们可以得到每一款车的平均售价、最低最高售价：
"aggregations" : {
"avage price per model" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "honda",
"doc_count" : ,
"max price" : {
"value" : 20000.0
},
"average price" : {
"value" : 16666.666666666668
},
"min price" : {
"value" : 10000.0
}
},
{
"key" : "ford",
"doc_count" : ,
"max price" : {
"value" : 30000.0
},
"average price" : {
"value" : 27500.0
},
"min price" : {
"value" : 25000.0
}
},
{
"key" : "toyota",
"doc_count" : ,
"max price" : {
"value" : 15000.0
},
"average price" : {
"value" : 13500.0
},
"min price" : {
"value" : 12000.0
}
},
{
"key" : "bmw",
"doc_count" : ,
"max price" : {
"value" : 80000.0
},
"average price" : {
"value" : 80000.0
},
"min price" : {
"value" : 80000.0
}
}
]
}
}
elastic4s示范如下：
val aggDisc = search("cartxns").aggregations(
termsAgg("prices_per_model","make.keyword").subAggregations(
avgAgg("average_price","price"),
minAgg("min_price","price"),
maxAgg("max_price","price")
)
)
println(aggDisc.show)
val discResult = client.execute(aggDisc).await
if (discResult.isSuccess)
discResult.result.aggregations.terms("prices_per_model").buckets
.foreach(mb =>
println(s"${mb.key},${mb.docCount}:${mb.avg("average_price").value}," +
s"${mb.min("min_price").value.getOrElse(0)}," +
s"${mb.max("max_price").value.getOrElse(0)}"))
else println(s"error: ${discResult.error.causedBy.getOrElse("unknown")}")
...
POST:/cartxns/_search?
StringEntity({"aggs":{"prices_per_model":{"terms":{"field":"make.keyword"},"aggs":{"average_price":{"avg":{"field":"price"}},"min_price":{"min":{"field":"price"}},"max_price":{"max":{"field":"price"}}}}}},Some(application/json))
honda,:16666.666666666668,10000.0,20000.0
ford,:27500.0,25000.0,30000.0
toyota,:13500.0,12000.0,15000.0
bmw,:80000.0,80000.0,80000.0
date_histogram是一种按时间间隔聚合的统计方法。对于按时间趋势变化的数据分析十分有用：
POST /cartxns/_search
{
"aggs": {
"sales_per_month": {
"date_histogram": {
"field": "sold",
"calendar_interval":"1M",
"format": "yyyy-MM-dd"
}
}
}
} ... "aggregations" : {
"sales_per_month" : {
"buckets" : [
{
"key_as_string" : "2014-01-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-02-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-03-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-04-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-05-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-06-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-07-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-08-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-09-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-10-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-11-01",
"key" : ,
"doc_count" :
}
]
}
}
上面这个例子产生以月为单元的bucket。elastic4s示范：
val aggDateHist = search("cartxns").aggregations(
dateHistogramAggregation("sales_per_month")
.field("sold")
.calendarInterval(DateHistogramInterval.Month)
.format("yyyy-MM-dd")
.minDocCount()
)
println(aggDateHist.show)
val dtHistResult = client.execute(aggDateHist).await
if (dtHistResult.isSuccess)
dtHistResult.result.aggregations.dateHistogram("sales_per_month").buckets
.foreach(db => println(s"${db.date},${db.docCount}"))
else println(s"error: ${dtHistResult.error.causedBy.getOrElse("unknown")}")
...
POST:/cartxns/_search?
StringEntity({"aggs":{"sales_per_month":{"date_histogram":{"calendar_interval":"1M","min_doc_count":,"format":"yyyy-MM-dd","field":"sold"}}}},Some(application/json))
--,
--,
--,
--,
--,
--,
--,
在以月划分bucket后可以再进行每个月的深度聚合：
POST /cartxns/_search
{
"aggs": {
"sales_per_month": {
"date_histogram": {
"field": "sold",
"calendar_interval":"1M",
"format": "yyyy-MM-dd"
},
"aggs": {
"per_make_sum": {
"terms": {
"field": "make.keyword",
"size":
},
"aggs": {
"sum_price": {
"sum": {"field": "price"}
}
}
},
"total_sum": {
"sum": {
"field": "price"
}
}
}
}
}
}
我们可以得到每个月的销售总额、每个车款每个月的销售，如下：
"aggregations" : {
"sales_per_month" : {
"buckets" : [
{
"key_as_string" : "2014-01-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "bmw",
"doc_count" : ,
"sum_price" : {
"value" : 80000.0
}
}
]
},
"total_sum" : {
"value" : 80000.0
}
},
{
"key_as_string" : "2014-02-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "ford",
"doc_count" : ,
"sum_price" : {
"value" : 25000.0
}
}
]
},
"total_sum" : {
"value" : 25000.0
}
},
{
"key_as_string" : "2014-03-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [ ]
},
"total_sum" : {
"value" : 0.0
}
},
{
"key_as_string" : "2014-04-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [ ]
},
"total_sum" : {
"value" : 0.0
}
},
{
"key_as_string" : "2014-05-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "ford",
"doc_count" : ,
"sum_price" : {
"value" : 30000.0
}
}
]
},
"total_sum" : {
"value" : 30000.0
}
},
{
"key_as_string" : "2014-06-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [ ]
},
"total_sum" : {
"value" : 0.0
}
},
{
"key_as_string" : "2014-07-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "toyota",
"doc_count" : ,
"sum_price" : {
"value" : 15000.0
}
}
]
},
"total_sum" : {
"value" : 15000.0
}
},
{
"key_as_string" : "2014-08-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "toyota",
"doc_count" : ,
"sum_price" : {
"value" : 12000.0
}
}
]
},
"total_sum" : {
"value" : 12000.0
}
},
{
"key_as_string" : "2014-09-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [ ]
},
"total_sum" : {
"value" : 0.0
}
},
{
"key_as_string" : "2014-10-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "honda",
"doc_count" : ,
"sum_price" : {
"value" : 10000.0
}
}
]
},
"total_sum" : {
"value" : 10000.0
}
},
{
"key_as_string" : "2014-11-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "honda",
"doc_count" : ,
"sum_price" : {
"value" : 40000.0
}
}
]
},
"total_sum" : {
"value" : 40000.0
}
}
]
}
}
用elastic4s可以这样写：
val aggMonthSales= search("cartxns").aggregations(
dateHistogramAggregation("sales_per_month")
.field("sold")
.calendarInterval(DateHistogramInterval.Month)
.format("yyyy-MM-dd")
.minDocCount().subAggregations(
termsAgg("month_make","make.keyword").subAggregations(
sumAggregation("month_total_per_make").field("price")
),
sumAggregation("monthly_total").field("price")
)
)
println(aggMonthSales.show)
val monthSalesResult = client.execute(aggMonthSales).await
if (monthSalesResult.isSuccess)
monthSalesResult.result.aggregations.dateHistogram("sales_per_month").buckets
.foreach { sb =>
println(s"${sb.date},${sb.docCount},${sb.sum("monthly_total").value}")
sb.terms("month_make").buckets
.foreach(mb =>
println(s"${mb.key},${mb.docCount},${mb.sum("month_total_per_make").value}"))
}
else println(s"error: ${monthSalesResult.error.causedBy.getOrElse("unknown")}")
...
POST:/cartxns/_search?
StringEntity({"aggs":{"sales_per_month":{"date_histogram":{"calendar_interval":"1M","min_doc_count":,"format":"yyyy-MM-dd","field":"sold"},"aggs":{"month_make":{"terms":{"field":"make.keyword"},"aggs":{"month_total_per_make":{"sum":{"field":"price"}}}},"monthly_total":{"sum":{"field":"price"}}}}}},Some(application/json))
--,,80000.0
bmw,,80000.0
--,,25000.0
ford,,25000.0
--,,30000.0
ford,,30000.0
--,,15000.0
toyota,,15000.0
--,,12000.0
toyota,,12000.0
--,,10000.0
honda,,10000.0
--,,40000.0
honda,,40000.0
search（13）- elastic4s-histograms：聚合直方图的更多相关文章
- 13 Tensorflow API主要功能
要想使用Tensorflow API,首先要知道它能干什么.Tensorflow具有Python.C++.Java.Go等多种语言API,其中Python的API是最简单和好用的. Tensor Tr ...
- TensorBoard中HISTOGRAMS和DISTRIBUTIONS图形的含义
前言 之前我都是用TensorBoard记录训练过程中的Loss.mAP等标量,很容易就知道TensorBoard里的SCALARS(标量)(其中横纵轴的含义.Smoothing等). 最近在尝试模型 ...
- Elasticsearch 2.3.3 JAVA api说明文档
原文地址:https://www.blog-china.cn/template\documentHtml\1484101683485.html 翻译作者:@青山常在人不老 加入翻译:cdcnsuper ...
- elasticsearch系列七：ES Java客户端-Elasticsearch Java client（ES Client 简介、Java REST Client、Java Client、Spring Data Elasticsearch）
一.ES Client 简介 1. ES是一个服务,采用C/S结构 2. 回顾 ES的架构 3. ES支持的客户端连接方式 3.1 REST API ,端口 9200 这种连接方式对应于架构图中的RE ...
- 【转载】DRuid 大数据分析之查询
转载自http://yangyangmyself.iteye.com/blog/2321759 1.Druid 查询概述 上一节完成数据导入后,接下来讲讲Druid如何查询及统计分析导入的数据 ...
- Elasticsearch Java client（ES Client 简介、Java REST Client、Java Client、Spring Data Elasticsearch）
elasticsearch系列七:ES Java客户端-Elasticsearch Java client(ES Client 简介.Java REST Client.Java Client.Spri ...
- 微服务监控之二：Metrics+influxdb+grafana构建监控平台
系统开发到一定的阶段,线上的机器越来越多,就需要一些监控了,除了服务器的监控,业务方面也需要一些监控服务.Metrics作为一款监控指标的度量类库,提供了许多工具帮助开发者来完成自定义的监控工作. 使 ...
- Elasticsearch技术解析与实战 PDF （内含目录）
Elasticsearch技术解析与实战 介绍: Elasticsearch是一个强[0大0]的搜索引擎,提供了近实时的索引.搜索.分 ...
- ML面试1000题系列（71-80）
本文总结ML面试常见的问题集 转载来源:https://blog.csdn.net/v_july_v/article/details/78121924 71.看你是搞视觉的,熟悉哪些CV框架,顺带聊聊 ...
随机推荐
- 刚从一道题发现的一些东西，PHP笔记,关于extract和null 空字符串
队友发给我的一道extract变量的最基础的题目,他发现了一些问题,当传入shiyan=&flag=0时出flag,当传入shiyan=0&flag=0时不出flag,传入shiyan ...
- JDBC教程——检视阅读
JDBC教程--检视阅读 参考 JDBC教程--W3Cschool JDBC教程--一点教程,有高级部分 JDBC教程--易百 JDBC入门教程 – 终极指南 略读 三层架构详解,JDBC在数据访问层 ...
- python 进阶篇 迭代器和生成器深入理解
列表/元组/字典/集合都是容器.对于容器,可以很直观地想象成多个元素在一起的单元:而不同容器的区别,正是在于内部数据结构的实现方法. 所有的容器都是可迭代的(iterable).另外字符串也可以被迭代 ...
- Java中接口的概念
接口的特点: A:接口用关键字interface表示 interface 接口名 {} B:类实现接口用 implements 表示 class 类名 implements 接口名 {} C:接口不能 ...
- SK-learn实现k近邻算法【准确率随k值的变化】-------莺尾花种类预测
代码详解: from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split fr ...
- 图解Python的垃圾回收机制
Python的GC模块主要运用了“引用计数”(reference counting)来跟踪和回收垃圾.在引用计数的基础上,还可以通过“标记-清除”(mark and sweep)解决容器对象可能产生的 ...
- 编程是要偷懒的--option简练写法
没改前: if(!empty($search)){ $where['personal_name'] = array('like','%'. $search . '%'); $this -> as ...
- Bogon
Definition - What does Bogon mean? A bogon is an bogus IP address from the bogon space, which is a s ...
- POJ1651:Multiplication Puzzle（区间dp）
Multiplication Puzzle Time Limit: 1000MS Memory Limit: 65536K Total Submissions: 9419 Accepted: 5850 ...
- windows右键没有新建选项的解决办法
1 以管理员身份运行cmd 2 cmd /k reg add "HKEY_CLASSES_ROOT\Directory\Background\shellex\ContextMenuHandl ...