search（13）- elastic4s-histograms：聚合直方图
在聚合的分组统计中我们会面临两种分组元素类型：连续型如时间，自然数等、离散型如地点、产品等。离散型数据本身就代表不同的组别，但连续型数据则需要手工按等长间隔进行切分了。下面是一个按价钱段聚合的例子：
POST /cartxns/_search
{
"size" : ,
"aggs": {
"sales_per_pricerange": {
"histogram": {
"field": "price",
"interval":
},
"aggs": {
"total sales": {
"sum": {
"field": "price"
}
}
}
}
}
}
}
在上面这个例子中我们把价钱按20000进行分段。得出0-19999,20000-39999,40000-59999 ... 价格段的度量：
"aggregations" : {
"sales_per_pricerange" : {
"buckets" : [
{
"key" : 0.0,
"doc_count" : ,
"total sales" : {
"value" : 37000.0
}
},
{
"key" : 20000.0,
"doc_count" : ,
"total sales" : {
"value" : 95000.0
}
},
{
"key" : 40000.0,
"doc_count" : ,
"total sales" : {
"value" : 0.0
}
},
{
"key" : 60000.0,
"doc_count" : ,
"total sales" : {
"value" : 0.0
}
},
{
"key" : 80000.0,
"doc_count" : ,
"total sales" : {
"value" : 80000.0
}
}
]
}
}
在elastic4s中是这样表达的：
val aggHist = search("cartxns").aggregations(
histogramAggregation("sales_per_price")
.field("price")
.interval().subAggregations(
sumAggregation("total_sales").field("price")
)
)
println(aggHist.show)
val histResult = client.execute(aggHist).await
if (histResult.isSuccess)
histResult.result.aggregations.histogram("sales_per_price").buckets
.foreach(hb => println(s"${hb.key},${hb.docCount}:${hb.sum("total_sales").value}"))
else println(s"error: ${histResult.error.reason}")
....
POST:/cartxns/_search?
StringEntity({"aggs":{"sales_per_price":{"histogram":{"interval":20000.0,"field":"price"},"aggs":{"total_sales":{"sum":{"field":"price"}}}}}},Some(application/json))
0.0,:37000.0
20000.0,:95000.0
40000.0,:0.0
60000.0,:0.0
80000.0,:80000.0
下面这个按车款分组统计的就是一个离散元素的聚合统计了：
POST /cartxns/_search
{
"size" : ,
"aggs": {
"avage price per model" : {
"terms": {"field" : "make.keyword"},
"aggs": {
"average price": {
"avg": {"field": "price"}
},
"max price" : {
"max": {
"field": "price"
}
},
"min price" : {
"min": {
"field": "price"
}
} }
}
}
}
我们可以得到每一款车的平均售价、最低最高售价：
"aggregations" : {
"avage price per model" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "honda",
"doc_count" : ,
"max price" : {
"value" : 20000.0
},
"average price" : {
"value" : 16666.666666666668
},
"min price" : {
"value" : 10000.0
}
},
{
"key" : "ford",
"doc_count" : ,
"max price" : {
"value" : 30000.0
},
"average price" : {
"value" : 27500.0
},
"min price" : {
"value" : 25000.0
}
},
{
"key" : "toyota",
"doc_count" : ,
"max price" : {
"value" : 15000.0
},
"average price" : {
"value" : 13500.0
},
"min price" : {
"value" : 12000.0
}
},
{
"key" : "bmw",
"doc_count" : ,
"max price" : {
"value" : 80000.0
},
"average price" : {
"value" : 80000.0
},
"min price" : {
"value" : 80000.0
}
}
]
}
}
elastic4s示范如下：
val aggDisc = search("cartxns").aggregations(
termsAgg("prices_per_model","make.keyword").subAggregations(
avgAgg("average_price","price"),
minAgg("min_price","price"),
maxAgg("max_price","price")
)
)
println(aggDisc.show)
val discResult = client.execute(aggDisc).await
if (discResult.isSuccess)
discResult.result.aggregations.terms("prices_per_model").buckets
.foreach(mb =>
println(s"${mb.key},${mb.docCount}:${mb.avg("average_price").value}," +
s"${mb.min("min_price").value.getOrElse(0)}," +
s"${mb.max("max_price").value.getOrElse(0)}"))
else println(s"error: ${discResult.error.causedBy.getOrElse("unknown")}")
...
POST:/cartxns/_search?
StringEntity({"aggs":{"prices_per_model":{"terms":{"field":"make.keyword"},"aggs":{"average_price":{"avg":{"field":"price"}},"min_price":{"min":{"field":"price"}},"max_price":{"max":{"field":"price"}}}}}},Some(application/json))
honda,:16666.666666666668,10000.0,20000.0
ford,:27500.0,25000.0,30000.0
toyota,:13500.0,12000.0,15000.0
bmw,:80000.0,80000.0,80000.0
date_histogram是一种按时间间隔聚合的统计方法。对于按时间趋势变化的数据分析十分有用：
POST /cartxns/_search
{
"aggs": {
"sales_per_month": {
"date_histogram": {
"field": "sold",
"calendar_interval":"1M",
"format": "yyyy-MM-dd"
}
}
}
} ... "aggregations" : {
"sales_per_month" : {
"buckets" : [
{
"key_as_string" : "2014-01-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-02-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-03-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-04-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-05-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-06-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-07-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-08-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-09-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-10-01",
"key" : ,
"doc_count" :
},
{
"key_as_string" : "2014-11-01",
"key" : ,
"doc_count" :
}
]
}
}
上面这个例子产生以月为单元的bucket。elastic4s示范：
val aggDateHist = search("cartxns").aggregations(
dateHistogramAggregation("sales_per_month")
.field("sold")
.calendarInterval(DateHistogramInterval.Month)
.format("yyyy-MM-dd")
.minDocCount()
)
println(aggDateHist.show)
val dtHistResult = client.execute(aggDateHist).await
if (dtHistResult.isSuccess)
dtHistResult.result.aggregations.dateHistogram("sales_per_month").buckets
.foreach(db => println(s"${db.date},${db.docCount}"))
else println(s"error: ${dtHistResult.error.causedBy.getOrElse("unknown")}")
...
POST:/cartxns/_search?
StringEntity({"aggs":{"sales_per_month":{"date_histogram":{"calendar_interval":"1M","min_doc_count":,"format":"yyyy-MM-dd","field":"sold"}}}},Some(application/json))
--,
--,
--,
--,
--,
--,
--,
在以月划分bucket后可以再进行每个月的深度聚合：
POST /cartxns/_search
{
"aggs": {
"sales_per_month": {
"date_histogram": {
"field": "sold",
"calendar_interval":"1M",
"format": "yyyy-MM-dd"
},
"aggs": {
"per_make_sum": {
"terms": {
"field": "make.keyword",
"size":
},
"aggs": {
"sum_price": {
"sum": {"field": "price"}
}
}
},
"total_sum": {
"sum": {
"field": "price"
}
}
}
}
}
}
我们可以得到每个月的销售总额、每个车款每个月的销售，如下：
"aggregations" : {
"sales_per_month" : {
"buckets" : [
{
"key_as_string" : "2014-01-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "bmw",
"doc_count" : ,
"sum_price" : {
"value" : 80000.0
}
}
]
},
"total_sum" : {
"value" : 80000.0
}
},
{
"key_as_string" : "2014-02-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "ford",
"doc_count" : ,
"sum_price" : {
"value" : 25000.0
}
}
]
},
"total_sum" : {
"value" : 25000.0
}
},
{
"key_as_string" : "2014-03-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [ ]
},
"total_sum" : {
"value" : 0.0
}
},
{
"key_as_string" : "2014-04-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [ ]
},
"total_sum" : {
"value" : 0.0
}
},
{
"key_as_string" : "2014-05-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "ford",
"doc_count" : ,
"sum_price" : {
"value" : 30000.0
}
}
]
},
"total_sum" : {
"value" : 30000.0
}
},
{
"key_as_string" : "2014-06-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [ ]
},
"total_sum" : {
"value" : 0.0
}
},
{
"key_as_string" : "2014-07-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "toyota",
"doc_count" : ,
"sum_price" : {
"value" : 15000.0
}
}
]
},
"total_sum" : {
"value" : 15000.0
}
},
{
"key_as_string" : "2014-08-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "toyota",
"doc_count" : ,
"sum_price" : {
"value" : 12000.0
}
}
]
},
"total_sum" : {
"value" : 12000.0
}
},
{
"key_as_string" : "2014-09-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [ ]
},
"total_sum" : {
"value" : 0.0
}
},
{
"key_as_string" : "2014-10-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "honda",
"doc_count" : ,
"sum_price" : {
"value" : 10000.0
}
}
]
},
"total_sum" : {
"value" : 10000.0
}
},
{
"key_as_string" : "2014-11-01",
"key" : ,
"doc_count" : ,
"per_make_sum" : {
"doc_count_error_upper_bound" : ,
"sum_other_doc_count" : ,
"buckets" : [
{
"key" : "honda",
"doc_count" : ,
"sum_price" : {
"value" : 40000.0
}
}
]
},
"total_sum" : {
"value" : 40000.0
}
}
]
}
}
用elastic4s可以这样写：
val aggMonthSales= search("cartxns").aggregations(
dateHistogramAggregation("sales_per_month")
.field("sold")
.calendarInterval(DateHistogramInterval.Month)
.format("yyyy-MM-dd")
.minDocCount().subAggregations(
termsAgg("month_make","make.keyword").subAggregations(
sumAggregation("month_total_per_make").field("price")
),
sumAggregation("monthly_total").field("price")
)
)
println(aggMonthSales.show)
val monthSalesResult = client.execute(aggMonthSales).await
if (monthSalesResult.isSuccess)
monthSalesResult.result.aggregations.dateHistogram("sales_per_month").buckets
.foreach { sb =>
println(s"${sb.date},${sb.docCount},${sb.sum("monthly_total").value}")
sb.terms("month_make").buckets
.foreach(mb =>
println(s"${mb.key},${mb.docCount},${mb.sum("month_total_per_make").value}"))
}
else println(s"error: ${monthSalesResult.error.causedBy.getOrElse("unknown")}")
...
POST:/cartxns/_search?
StringEntity({"aggs":{"sales_per_month":{"date_histogram":{"calendar_interval":"1M","min_doc_count":,"format":"yyyy-MM-dd","field":"sold"},"aggs":{"month_make":{"terms":{"field":"make.keyword"},"aggs":{"month_total_per_make":{"sum":{"field":"price"}}}},"monthly_total":{"sum":{"field":"price"}}}}}},Some(application/json))
--,,80000.0
bmw,,80000.0
--,,25000.0
ford,,25000.0
--,,30000.0
ford,,30000.0
--,,15000.0
toyota,,15000.0
--,,12000.0
toyota,,12000.0
--,,10000.0
honda,,10000.0
--,,40000.0
honda,,40000.0
search（13）- elastic4s-histograms：聚合直方图的更多相关文章
- 13 Tensorflow API主要功能
要想使用Tensorflow API,首先要知道它能干什么.Tensorflow具有Python.C++.Java.Go等多种语言API,其中Python的API是最简单和好用的. Tensor Tr ...
- TensorBoard中HISTOGRAMS和DISTRIBUTIONS图形的含义
前言 之前我都是用TensorBoard记录训练过程中的Loss.mAP等标量,很容易就知道TensorBoard里的SCALARS(标量)(其中横纵轴的含义.Smoothing等). 最近在尝试模型 ...
- Elasticsearch 2.3.3 JAVA api说明文档
原文地址:https://www.blog-china.cn/template\documentHtml\1484101683485.html 翻译作者:@青山常在人不老 加入翻译:cdcnsuper ...
- elasticsearch系列七：ES Java客户端-Elasticsearch Java client（ES Client 简介、Java REST Client、Java Client、Spring Data Elasticsearch）
一.ES Client 简介 1. ES是一个服务,采用C/S结构 2. 回顾 ES的架构 3. ES支持的客户端连接方式 3.1 REST API ,端口 9200 这种连接方式对应于架构图中的RE ...
- 【转载】DRuid 大数据分析之查询
转载自http://yangyangmyself.iteye.com/blog/2321759 1.Druid 查询概述 上一节完成数据导入后,接下来讲讲Druid如何查询及统计分析导入的数据 ...
- Elasticsearch Java client（ES Client 简介、Java REST Client、Java Client、Spring Data Elasticsearch）
elasticsearch系列七:ES Java客户端-Elasticsearch Java client(ES Client 简介.Java REST Client.Java Client.Spri ...
- 微服务监控之二：Metrics+influxdb+grafana构建监控平台
系统开发到一定的阶段,线上的机器越来越多,就需要一些监控了,除了服务器的监控,业务方面也需要一些监控服务.Metrics作为一款监控指标的度量类库,提供了许多工具帮助开发者来完成自定义的监控工作. 使 ...
- Elasticsearch技术解析与实战 PDF （内含目录）
Elasticsearch技术解析与实战 介绍: Elasticsearch是一个强[0大0]的搜索引擎,提供了近实时的索引.搜索.分 ...
- ML面试1000题系列（71-80）
本文总结ML面试常见的问题集 转载来源:https://blog.csdn.net/v_july_v/article/details/78121924 71.看你是搞视觉的,熟悉哪些CV框架,顺带聊聊 ...
随机推荐
- 详解 JDK8 新增的日期时间类
JDK8 新增的日期时间类 在本人之前的博文<处理时间的类 -- System类.Date类 .SimpleDateFormat类 与 Calendar类>中,讲到过表示时间的类,有三类: ...
- Linux工程师必备的系统监控工具
WGCLOUD基于java语言开发,是微服务架构构建监控系统,支持高并发高性能高可用,核心模块包括:服务器集群监控,ES集群状态监控,CPU监控,内存监控,数据监控(mysql,postgresql, ...
- pytorch 去除维度为1的维度
out.squeeze(dim=1) out.squeeze_(dim=1)
- pytorch Dataset数据集和Dataloader迭代数据集
import torch from torch.utils.data import Dataset,DataLoader class SmsDataset(Dataset): def __init__ ...
- Windows 挂起进程
A thread can suspend and resume the execution of another thread. While a thread is suspended, it is ...
- java在指定区间内生成随机数
Random对象生成随机数 首先需要导入包含Random的包 import java.util.Random; nextInt(int)方法将生成0~参数之间的随机整数但不包括参数. 例如生成0~99 ...
- 【Linux题目】第九关
前言:项目整合 企业项目实战考试: 1. 全网备份解决方案实战 2. NFS集群后段共享存储搭建优化 3. 解决NFS单点实现实时数据同步. 环境: 服务器角色 外网ip 内网ip 主机名 web 1 ...
- mac OS mysql新建数据库运行sql文件
mysql -uroot -proot123 进入本地数据库 create database 数据库名; use 数据库名; source 文件路径 quit 退出
- 信息竞赛进阶指南--递归法求中缀表达式的值，O(n^2)（模板）
// 递归法求中缀表达式的值,O(n^2) int calc(int l, int r) { // 寻找未被任何括号包含的最后一个加减号 for (int i = r, j = 0; i >= ...
- UTC 时间转化为北京时间
// UTC 时间转化为北京时间 function utc2beijing(utcTime) { var T_pos = utcTime.indexOf('T'); var Z_pos = utcTi ...