ElasticSearch6.x版本聚合分析整理
ElasticSearch6.x版本聚合分析整理
ES将聚合分析主要分为如下4类
- Bucket,分桶类型,类似SQL中的GROUP BY语法
- Metric,指标分析类型,如计算最大值 , 最小值,平均值等
- Pipeline,管道分析类型,基于上一级的聚合分析结果进行再分析
- Matrix,矩阵分析类型
Metric聚合分析
主要分如下两类:
1.单值分析,只输出一个分析结果
min,max,avg,sum
cardinality
2.多值分析,输出多个分析结果
stats,extended stats
percentile,percentile rank
top hits
需要使用到的数据:
POST test_search_index/doc/_bulk
{"index":{"_id":"1"}}
{"username":"alfred way","job":"java engineer","age":18,"birth":"1990-01-02","isMarried":false,"salary":10000}
{"index":{"_id":"2"}}
{"username":"tom","job":"java senior engineer","age":28,"birth":"1980-05-07","isMarried":true,"salary":30000}
{"index":{"_id":"3"}}
{"username":"lee","job":"ruby engineer","age":22,"birth":"1985-08-07","isMarried":false,"salary":15000}
{"index":{"_id":"4"}}
{"username":"Nick","job":"web engineer","age":23,"birth":"1989-08-07","isMarried":false,"salary":8000}
{"index":{"_id":"5"}}
{"username":"Niko","job":"web engineer","age":18,"birth":"1994-08-07","isMarried":false,"salary":5000}
{"index":{"_id":"6"}}
{"username":"Michell","job":"ruby engineer","age":26,"birth":"1987-08-07","isMarried":false,"salary":12000}
Metric聚合分析
返回数值类字段的平均值
GET test_search_index/_search
{
#不需要返回文档列表
"size":0,
"aggs":{
#名字
"min_age":{
#关键词
"min":{
"field":"age"
}
}
}
}
#返回结果:
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"min_age": {
"value": 18
}
}
}
#返回数值类字段的最大值
GET test_search_index/_search
{
"size":0,
"aggs":{
"max_age":{
"max":{
"field":"age"
}
}
}
}
#返回结果
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"max_age": {
"value": 28
}
}
}
#返回数值类字段的平均值
GET test_search_index/_search
{
"size":0,
"aggs":{
"avg_age":{
"avg":{
"field":"age"
}
}
}
}
#返回结果
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"avg_age": {
"value": 22.5
}
}
}
#返回数值字段的总和
GET test_search_index/_search
{
"size":0,
"aggs":{
"sum_age":{
"sum":{
"field":"age"
}
}
}
}
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"sum_age": {
"value": 135
}
}
}
#一次返回多个结果
{
"size":0,
"aggs":{
"min_age":{
"min":{
"field":"age"
}
},
"max_age":{
"max":{
"field":"age"
}
},
"sum_age":{
"sum":{
"field":"age"
}
}
}
}
#返回结果
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"max_age": {
"value": 28
},
"sum_age": {
"value": 135
},
"min_age": {
"value": 18
}
}
}
Metric聚合分析--Cardinality
Cardinality,意为集合的势,或者基数,是指不同数值的个数,类似SQL中的distinct count概念
GET test_search_index/_search
{
"size":10,
"aggs":{
"count_of_job":{
"cardinality":{
"field":"job.keyword"
}
}
}
}
#返回
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 1,
"hits": [
{
"_index": "test_search_index",
"_type": "doc",
"_id": "5",
"_score": 1,
"_source": {
"username": "Niko",
"job": "web engineer",
"age": 18,
"birth": "1994-08-07",
"isMarried": false,
"salary": 5000
}
},
{
"_index": "test_search_index",
"_type": "doc",
"_id": "2",
"_score": 1,
"_source": {
"username": "tom",
"job": "java senior engineer",
"age": 28,
"birth": "1980-05-07",
"isMarried": true,
"salary": 30000
}
},
{
"_index": "test_search_index",
"_type": "doc",
"_id": "4",
"_score": 1,
"_source": {
"username": "Nick",
"job": "web engineer",
"age": 23,
"birth": "1989-08-07",
"isMarried": false,
"salary": 8000
}
},
{
"_index": "test_search_index",
"_type": "doc",
"_id": "6",
"_score": 1,
"_source": {
"username": "Michell",
"job": "ruby engineer",
"age": 26,
"birth": "1987-08-07",
"isMarried": false,
"salary": 12000
}
},
{
"_index": "test_search_index",
"_type": "doc",
"_id": "1",
"_score": 1,
"_source": {
"username": "alfred way",
"job": "java engineer",
"age": 18,
"birth": "1990-01-02",
"isMarried": false,
"salary": 10000
}
},
{
"_index": "test_search_index",
"_type": "doc",
"_id": "3",
"_score": 1,
"_source": {
"username": "lee",
"job": "ruby engineer",
"age": 22,
"birth": "1985-08-07",
"isMarried": false,
"salary": 15000
}
}
]
},
"aggregations": {
"count_of_job": {
"value": 4
}
}
}
Metric聚合分析-Stats
返回一系列数值类型的统计值,包含min,max,avg,sum和count
GET test_search_index/_search
{
"size":0,
"aggs":{
"stats_age":{
"stats":{
"field":"age"
}
}
}
}
#返回
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"stats_age": {
"count": 6,
"min": 18,
"max": 28,
"avg": 22.5,
"sum": 135
}
}
}
Metric聚合分析-Extended Stats
对stats的扩展,包含了更多的统计数据,如方差,标准差等
GET test_search_index/_search
{
"size":0,
"aggs":{
"stats_age":{
"extended_stats":{
"field":"age"
}
}
}
}
#返回
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"stats_age": {
"count": 6,
"min": 18,
"max": 28,
"avg": 22.5,
"sum": 135,
"sum_of_squares": 3121,
"variance": 13.916666666666666,
"std_deviation": 3.730504880933232,
"std_deviation_bounds": {
"upper": 29.961009761866464,
"lower": 15.038990238133536
}
}
}
}
Metric聚合分析-Percentile
百分位数统计
GET test_search_index/_search
{
"size":0,
"aggs":{
"per_age":{
"percentiles":{
"field":"salary"
}
}
}
}
#返回
{
"took": 6,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"per_age": {
#代表有百分之一的人工资在5000以下,百分之二十五的人工资在8000以下....
"values": {
"1.0": 5000,
"5.0": 5000,
"25.0": 8000,
"50.0": 11000,
"75.0": 15000,
"95.0": 30000,
"99.0": 30000
}
}
}
}
GET test_search_index/_search
{
"size":0,
"aggs":{
"per_age":{
"percentile_ranks":{
"field":"salary",
"values":[
11000,
30000
]
}
}
}
}
#返回
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"per_age": {
"values": {
"11000.0": 50,
"30000.0": 100
}
}
}
}
Metric聚合分析-Top Hits
一般用于分桶后获取该桶内最匹配的顶部文档列表,即详情数据
#先按照job分桶, 然后在桶内做年龄的排序
GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
},
"aggs":{
"top_employee":{
"top_hits":{
"size":10,
"sort":[
{
"age":{
"order":"desc"
}
}
]
}
}
}
}
}
}
#返回
{
"took": 42,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"jobs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "ruby engineer",
"doc_count": 2,
"top_employee": {
"hits": {
"total": 2,
"max_score": null,
"hits": [
{
"_index": "test_search_index",
"_type": "doc",
"_id": "6",
"_score": null,
"_source": {
"username": "Michell",
"job": "ruby engineer",
"age": 26,
"birth": "1987-08-07",
"isMarried": false,
"salary": 12000
},
"sort": [
26
]
},
{
"_index": "test_search_index",
"_type": "doc",
"_id": "3",
"_score": null,
"_source": {
"username": "lee",
"job": "ruby engineer",
"age": 22,
"birth": "1985-08-07",
"isMarried": false,
"salary": 15000
},
"sort": [
22
]
}
]
}
}
},
{
"key": "web engineer",
"doc_count": 2,
"top_employee": {
"hits": {
"total": 2,
"max_score": null,
"hits": [
{
"_index": "test_search_index",
"_type": "doc",
"_id": "4",
"_score": null,
"_source": {
"username": "Nick",
"job": "web engineer",
"age": 23,
"birth": "1989-08-07",
"isMarried": false,
"salary": 8000
},
"sort": [
23
]
},
{
"_index": "test_search_index",
"_type": "doc",
"_id": "5",
"_score": null,
"_source": {
"username": "Niko",
"job": "web engineer",
"age": 18,
"birth": "1994-08-07",
"isMarried": false,
"salary": 5000
},
"sort": [
18
]
}
]
}
}
},
{
"key": "java engineer",
"doc_count": 1,
"top_employee": {
"hits": {
"total": 1,
"max_score": null,
"hits": [
{
"_index": "test_search_index",
"_type": "doc",
"_id": "1",
"_score": null,
"_source": {
"username": "alfred way",
"job": "java engineer",
"age": 18,
"birth": "1990-01-02",
"isMarried": false,
"salary": 10000
},
"sort": [
18
]
}
]
}
}
},
{
"key": "java senior engineer",
"doc_count": 1,
"top_employee": {
"hits": {
"total": 1,
"max_score": null,
"hits": [
{
"_index": "test_search_index",
"_type": "doc",
"_id": "2",
"_score": null,
"_source": {
"username": "tom",
"job": "java senior engineer",
"age": 28,
"birth": "1980-05-07",
"isMarried": true,
"salary": 30000
},
"sort": [
28
]
}
]
}
}
}
]
}
}
}
Bucket聚合分析
Terms
该分桶策略最简单,直接按照term来分桶,如果是text类型,则按照分次后的结果分桶
GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
}
}
}
}
#返回
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"jobs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "ruby engineer",
"doc_count": 2
},
{
"key": "web engineer",
"doc_count": 2
},
{
"key": "java engineer",
"doc_count": 1
},
{
"key": "java senior engineer",
"doc_count": 1
}
]
}
}
}
Range
通过制定数值的范围来设定分桶规则
GET test_search_index/_search
{
"size":0,
"aggs":{
"salary_range":{
"range":{
"field":"salary",
"ranges":[
{
"to":10000
},
{
"from":10000,
"to":20000
},
{
"from":20000
}
]
}
}
}
}
#返回
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"salary_range": {
"buckets": [
{
"key": "*-10000.0",
"to": 10000,
"doc_count": 2
},
{
"key": "10000.0-20000.0",
"from": 10000,
"to": 20000,
"doc_count": 3
},
{
"key": "20000.0-*",
"from": 20000,
"doc_count": 1
}
]
}
}
}
Date Range
通过制定日期的范围来设定分桶规则
GET test_search_index/_search
{
"size":0,
"aggs":{
"date_range":{
"range":{
"field":"birth",
"format":"yyyy",
"ranges":[
{
"from":"1980",
"to":"1990"
},
{
"from":"1990",
"to":"2000"
},
{
"from":"2000"
}
]
}
}
}
}
#返回
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"date_range": {
"buckets": [
{
"key": "1980-1990",
"from": 315532800000,
"from_as_string": "1980",
"to": 631152000000,
"to_as_string": "1990",
"doc_count": 4
},
{
"key": "1990-2000",
"from": 631152000000,
"from_as_string": "1990",
"to": 946684800000,
"to_as_string": "2000",
"doc_count": 2
},
{
"key": "2000-*",
"from": 946684800000,
"from_as_string": "2000",
"doc_count": 0
}
]
}
}
}
Historgram
直方图,以固定间隔的策略来分隔数据
#表示间隔5000分隔工资的分布情况, 最小0,最大40000
GET test_search_index/_search
{
"size":0,
"aggs":{
"salary_hist":{
"histogram":{
"field":"salary",
"interval":5000,
"extended_bounds":{
"min":0,
"max":40000
}
}
}
}
}
#返回
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"salary_hist": {
"buckets": [
{
"key": 0,
"doc_count": 0
},
{
"key": 5000,
"doc_count": 2
},
{
"key": 10000,
"doc_count": 2
},
{
"key": 15000,
"doc_count": 1
},
{
"key": 20000,
"doc_count": 0
},
{
"key": 25000,
"doc_count": 0
},
{
"key": 30000,
"doc_count": 1
},
{
"key": 35000,
"doc_count": 0
},
{
"key": 40000,
"doc_count": 0
}
]
}
}
}
Date Historgram
针对日期的直方图或者柱状图,是时序数据分析中常用的聚合分析类型
GET test_search_index/_search
{
"size":0,
"aggs":{
"salary_hist":{
"date_histogram":{
"field":"birth",
"interval":"year",
"format":"yyyy"
}
}
}
}
#返回
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"salary_hist": {
"buckets": [
{
"key_as_string": "1980",
"key": 315532800000,
"doc_count": 1
},
{
"key_as_string": "1981",
"key": 347155200000,
"doc_count": 0
},
{
"key_as_string": "1982",
"key": 378691200000,
"doc_count": 0
},
{
"key_as_string": "1983",
"key": 410227200000,
"doc_count": 0
},
{
"key_as_string": "1984",
"key": 441763200000,
"doc_count": 0
},
{
"key_as_string": "1985",
"key": 473385600000,
"doc_count": 1
},
{
"key_as_string": "1986",
"key": 504921600000,
"doc_count": 0
},
{
"key_as_string": "1987",
"key": 536457600000,
"doc_count": 1
},
{
"key_as_string": "1988",
"key": 567993600000,
"doc_count": 0
},
{
"key_as_string": "1989",
"key": 599616000000,
"doc_count": 1
},
{
"key_as_string": "1990",
"key": 631152000000,
"doc_count": 1
},
{
"key_as_string": "1991",
"key": 662688000000,
"doc_count": 0
},
{
"key_as_string": "1992",
"key": 694224000000,
"doc_count": 0
},
{
"key_as_string": "1993",
"key": 725846400000,
"doc_count": 0
},
{
"key_as_string": "1994",
"key": 757382400000,
"doc_count": 1
}
]
}
}
}
Bucket+Metric聚合分析
分桶后再分桶
1. GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
},
"aggs":{
"age_range":{
"range":{
"field":"age",
"ranges":[
{"to":20},
{"from":20,"to":30},
{"from":30}
]
}
}
}
}
}
}
#返回
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"jobs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "ruby engineer",
"doc_count": 2,
"age_range": {
"buckets": [
{
"key": "*-20.0",
"to": 20,
"doc_count": 0
},
{
"key": "20.0-30.0",
"from": 20,
"to": 30,
"doc_count": 2
},
{
"key": "30.0-*",
"from": 30,
"doc_count": 0
}
]
}
},
{
"key": "web engineer",
"doc_count": 2,
"age_range": {
"buckets": [
{
"key": "*-20.0",
"to": 20,
"doc_count": 1
},
{
"key": "20.0-30.0",
"from": 20,
"to": 30,
"doc_count": 1
},
{
"key": "30.0-*",
"from": 30,
"doc_count": 0
}
]
}
},
{
"key": "java engineer",
"doc_count": 1,
"age_range": {
"buckets": [
{
"key": "*-20.0",
"to": 20,
"doc_count": 1
},
{
"key": "20.0-30.0",
"from": 20,
"to": 30,
"doc_count": 0
},
{
"key": "30.0-*",
"from": 30,
"doc_count": 0
}
]
}
},
{
"key": "java senior engineer",
"doc_count": 1,
"age_range": {
"buckets": [
{
"key": "*-20.0",
"to": 20,
"doc_count": 0
},
{
"key": "20.0-30.0",
"from": 20,
"to": 30,
"doc_count": 1
},
{
"key": "30.0-*",
"from": 30,
"doc_count": 0
}
]
}
}
]
}
}
}
2.分桶后进行数据分析
GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
},
"aggs":{
"salary":{
"stats":{
"field":"salary"
}
}
}
}
}
}
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"jobs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "ruby engineer",
"doc_count": 2,
"salary": {
"count": 2,
"min": 12000,
"max": 15000,
"avg": 13500,
"sum": 27000
}
},
{
"key": "web engineer",
"doc_count": 2,
"salary": {
"count": 2,
"min": 5000,
"max": 8000,
"avg": 6500,
"sum": 13000
}
},
{
"key": "java engineer",
"doc_count": 1,
"salary": {
"count": 1,
"min": 10000,
"max": 10000,
"avg": 10000,
"sum": 10000
}
},
{
"key": "java senior engineer",
"doc_count": 1,
"salary": {
"count": 1,
"min": 30000,
"max": 30000,
"avg": 30000,
"sum": 30000
}
}
]
}
}
}
Pipeline聚合分析
针对聚合分析的结果再次进行聚合分析,而且支持链式调用
Pipeline的分析结果会输出到原结果中,根据输出位置的不同,分为以下两类:
1.Parent结果内嵌到现有的聚合分析结果中
Derivative
Moving Average
Cumulative Sum
2.Sibling结果与现有聚合分析结果同级
Max/Min/Avg/Sum Bucket
Stats/Extended Stats Bucket
Percentitles Bucket
Sibling - Min Bucket
找出所有Bucket中值最小的Bucket名称和值
1.聚合分析求job的和
2.job里面内嵌套了一个求平均salary
3.然后用一个同级的 Min Bucket求上面平均工资里面最小的那个
GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
},
"aggs":{
"avg_salary":{
"avg":{
"field":"salary"
}
}
}
},
"min_salary_by_job":{
"min_bucket":{
"buckets_path":"jobs>avg_salary"
}
}
}
}
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"jobs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "ruby engineer",
"doc_count": 2,
"avg_salary": {
"value": 13500
}
},
{
"key": "web engineer",
"doc_count": 2,
"avg_salary": {
"value": 6500
}
},
{
"key": "java engineer",
"doc_count": 1,
"avg_salary": {
"value": 10000
}
},
{
"key": "java senior engineer",
"doc_count": 1,
"avg_salary": {
"value": 30000
}
}
]
},
"min_salary_by_job": {
"value": 6500,
"keys": [
"web engineer"
]
}
}
}
找出所有Bucket中值最大的Bucket名称和值
GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
},
"aggs":{
"avg_salary":{
"avg":{
"field":"salary"
}
}
}
},
"max_salary_by_job":{
"max_bucket":{
"buckets_path":"jobs>avg_salary"
}
}
}
}
找出所有Bucket中值平均值
GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
},
"aggs":{
"avg_salary":{
"avg":{
"field":"salary"
}
}
}
},
"avg_salary_by_job":{
"avg_bucket":{
"buckets_path":"jobs>avg_salary"
}
}
}
}
计算所有Bucket值的Stats分析
GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
},
"aggs":{
"avg_salary":{
"avg":{
"field":"salary"
}
}
}
},
"stats_salary_by_job":{
"stats_bucket":{
"buckets_path":"jobs>avg_salary"
}
}
}
}
#返回
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"jobs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "ruby engineer",
"doc_count": 2,
"avg_salary": {
"value": 13500
}
},
{
"key": "web engineer",
"doc_count": 2,
"avg_salary": {
"value": 6500
}
},
{
"key": "java engineer",
"doc_count": 1,
"avg_salary": {
"value": 10000
}
},
{
"key": "java senior engineer",
"doc_count": 1,
"avg_salary": {
"value": 30000
}
}
]
},
"stats_salary_by_job": {
"count": 4,
"min": 6500,
"max": 30000,
"avg": 15000,
"sum": 60000
}
}
}
计算所有Bucket值的百分位数
GET test_search_index/_search
{
"size":0,
"aggs":{
"jobs":{
"terms":{
"field":"job.keyword",
"size":10
},
"aggs":{
"avg_salary":{
"avg":{
"field":"salary"
}
}
}
},
"percentiles_salary_by_job":{
"percentiles_bucket":{
"buckets_path":"jobs>avg_salary"
}
}
}
}
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"jobs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "ruby engineer",
"doc_count": 2,
"avg_salary": {
"value": 13500
}
},
{
"key": "web engineer",
"doc_count": 2,
"avg_salary": {
"value": 6500
}
},
{
"key": "java engineer",
"doc_count": 1,
"avg_salary": {
"value": 10000
}
},
{
"key": "java senior engineer",
"doc_count": 1,
"avg_salary": {
"value": 30000
}
}
]
},
"percentiles_salary_by_job": {
"values": {
"1.0": 6500,
"5.0": 6500,
"25.0": 10000,
"50.0": 13500,
"75.0": 13500,
"95.0": 30000,
"99.0": 30000
}
}
}
}
Parent- Derivative
计算Bucket值的导数
GET test_search_index/_search
{
"size":0,
"aggs":{
"birth":{
"date_histogram":{
"field":"birth",
"interval":"year",
"min_doc_count":0
},
"aggs":{
"avg_salary":{
"avg":{
"field":"salary"
}
},
"derivative_avg_salary":{
"derivative":{
"buckets_path":"avg_salary"
}
}
}
}
}
}
#返回
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": 0,
"hits": []
},
"aggregations": {
"birth": {
"buckets": [
{
"key_as_string": "1980-01-01T00:00:00.000Z",
"key": 315532800000,
"doc_count": 1,
"avg_salary": {
"value": 30000
}
},
{
"key_as_string": "1981-01-01T00:00:00.000Z",
"key": 347155200000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1982-01-01T00:00:00.000Z",
"key": 378691200000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1983-01-01T00:00:00.000Z",
"key": 410227200000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1984-01-01T00:00:00.000Z",
"key": 441763200000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1985-01-01T00:00:00.000Z",
"key": 473385600000,
"doc_count": 1,
"avg_salary": {
"value": 15000
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1986-01-01T00:00:00.000Z",
"key": 504921600000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1987-01-01T00:00:00.000Z",
"key": 536457600000,
"doc_count": 1,
"avg_salary": {
"value": 12000
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1988-01-01T00:00:00.000Z",
"key": 567993600000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1989-01-01T00:00:00.000Z",
"key": 599616000000,
"doc_count": 1,
"avg_salary": {
"value": 8000
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1990-01-01T00:00:00.000Z",
"key": 631152000000,
"doc_count": 1,
"avg_salary": {
"value": 10000
},
"derivative_avg_salary": {
"value": 2000
}
},
{
"key_as_string": "1991-01-01T00:00:00.000Z",
"key": 662688000000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1992-01-01T00:00:00.000Z",
"key": 694224000000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1993-01-01T00:00:00.000Z",
"key": 725846400000,
"doc_count": 0,
"avg_salary": {
"value": null
},
"derivative_avg_salary": {
"value": null
}
},
{
"key_as_string": "1994-01-01T00:00:00.000Z",
"key": 757382400000,
"doc_count": 1,
"avg_salary": {
"value": 5000
},
"derivative_avg_salary": {
"value": null
}
}
]
}
}
}
ElasticSearch6.x版本聚合分析整理的更多相关文章
- Elasticsearch 6.x版本全文检索学习之聚合分析入门
1.什么是聚合分析? 答:聚合分析,英文为Aggregation,是es除搜索功能外提供的针对es数据做统计分析的功能.特点如下所示: a.功能丰富,提供Bucket.Metric.Pipeline等 ...
- 根据版本的不同整理所有的绿色SQL Server
在这篇论坛文章中,读者可以了解到如何根据不同的SQL Server版本,整理出所有版本的绿色SQL Server的具体方法,详细内容请参考下文: 1. Sqlservr.exe 运行参数 Sql Se ...
- SQL语句优化技术分析 整理他人的
一.操作符优化 1.IN 操作符 用IN写出来的SQL的优点是比较容易写及清晰易懂,这比较适合现代软件开发的风格.但是用IN的SQL性能总是比较低的,从Oracle执行的步骤来分析用IN的SQL与不用 ...
- Elasticsearch学习笔记（三）聚合分析Agg
一.设置fielddata PUT /index/_mapping/type { "properties":{ "fieldName" ...
- ElasticSearch聚合分析
聚合用于分析查询结果集的统计指标,我们以观看日志分析为例,介绍各种常用的ElasticSearch聚合操作. 目录: 查询用户观看视频数和观看时长 聚合分页器 查询视频uv 单个视频uv 批量查询视频 ...
- element-ui 组件源码分析整理笔记目录
element-ui button组件 radio组件源码分析整理笔记(一) element-ui switch组件源码分析整理笔记(二) element-ui inputNumber.Card .B ...
- element-ui Carousel 走马灯源码分析整理笔记（十一）
Carousel 走马灯源码分析整理笔记,这篇写的不详细,后面有空补充 main.vue <template> <!--走马灯的最外层包裹div--> <div clas ...
- ES系列十四、ES聚合分析（聚合分析简介、指标聚合、桶聚合）
一.聚合分析简介 1. ES聚合分析是什么? 聚合分析是数据库中重要的功能特性,完成对一个查询的数据集中数据的聚合计算,如:找出某字段(或计算表达式的结果)的最大值.最小值,计算和.平均值等.ES作为 ...
- es-aggregations聚合分析
聚合分析的格式: "aggregations" : { "<aggregation_name>" : { "<aggregation ...
随机推荐
- K8s集群部署（二）------ Master节点部署
Master节点要部署三个服务:API Server.Scheduler.Controller Manager. apiserver提供集群管理的REST API接口,包括认证授权.数据校验以 及集群 ...
- 6tunnel数据转发
6tunnel 一条命令实现端口映射.数据转发,实现代理服务器功能. 安装脚本 #!/bin/bash DIR=/opt/software INSTALL=6tunnel-master.tar.gz ...
- Cisco packet tracer6.0下的网络工程实训
期末的专业实训,在cisco6.0下配置一个简单的局域网.主要用到了下面几个技术:dhcp中继.vlan的划分.链路聚合.静态nat.ospf协议.访问控制列表.先看一下总的拓扑图,在分步实现功能. ...
- ZOJ 3955：Saddle Point（思维）
http://acm.zju.edu.cn/onlinejudge/showProblem.do?problemCode=3955 题意:给出一个n*m的矩阵,定义矩阵中的特殊点Aij当且仅当Aij是 ...
- Ubuntu 16.4-desktop系统安装显卡CUDA具体步骤！
1.禁用nouveau驱动(切换至tty界面) sudo vim /etc/modprobe.d/blacklist.conf 在文本最后添加:blacklist nouveau options no ...
- django基础知识之HttpReqeust对象:
HttpReqeust对象 服务器接收到http协议的请求后,会根据报文创建HttpRequest对象 视图函数的第一个参数是HttpRequest对象 在django.http模块中定义了HttpR ...
- 2018年东北地区赛S - Problem I. Spell Boost HDU - 6508
题目地址:https://vjudge.net/problem/HDU-6508 思路:给一些卡,分为四种卡.1.白卡(没效果)2.魔法,作用卡(会对作用卡的费用减少,也会被魔法卡作用)3.作用卡(会 ...
- 作者联系方式D1
欢迎大伙投稿,审核通过免费发布. 奥特曼超人 KARL-Dujinyang QQ: 309933706 QQ: 1875125470 工作时间都会在线. 偶尔博客,不过有问题可以加Q来找我讨论 ...
- python接口自动化（三十）--html测试报告通过邮件发出去——中（详解）
简介 上一篇,我们虽然已经将生成的最新的测试报告发出去了,但是MIMEText 只能发送正文,无法带附件,因此我还需要继续改造我们的代码,实现可以发送带有附件的邮件.发送带附件的需要导入另外一个模块 ...
- Golang 受欢迎的原因：大道至简
前言 Golang自2009年发布第一个版本,2012年发布1.0版本.在这10年的时间里,不断有开发者加入Golang的阵营中,不断共建Golang生态.其中比较有代表性的Golang编写软件作品是 ...