50.percentiles百分比算法以及网站延时统计

主要知识点

• percentiles的用法

 
 

现有一个需求：比如有一个网站，记录下了每次请求的访问的耗时，需要统计tp50，tp90，tp99

• tp50：50%的请求的耗时最长在多长时间
• tp90：90%的请求的耗时最长在多长时间
• tp99：99%的请求的耗时最长在多长时间

 
 

一、准备数据

1、建立mappings

 
 

PUT /website

{

"mappings": {

"logs":{

"properties": {

"latency":{"type": "long"},

"province":{"type": "keyword"},

"timestamp":{"type":"date"}

}

}

}

}

2、批量插入数据

 
 

POST /website/logs/_bulk

{ "index": {}}

{ "latency" : 105, "province" : "江苏", "timestamp" : "2016-10-28" }

{ "index": {}}

{ "latency" : 83, "province" : "江苏", "timestamp" : "2016-10-29" }

{ "index": {}}

{ "latency" : 92, "province" : "江苏", "timestamp" : "2016-10-29" }

{ "index": {}}

{ "latency" : 112, "province" : "江苏", "timestamp" : "2016-10-28" }

{ "index": {}}

{ "latency" : 68, "province" : "江苏", "timestamp" : "2016-10-28" }

{ "index": {}}

{ "latency" : 76, "province" : "江苏", "timestamp" : "2016-10-29" }

{ "index": {}}

{ "latency" : 101, "province" : "新疆", "timestamp" : "2016-10-28" }

{ "index": {}}

{ "latency" : 275, "province" : "新疆", "timestamp" : "2016-10-29" }

{ "index": {}}

{ "latency" : 166, "province" : "新疆", "timestamp" : "2016-10-29" }

{ "index": {}}

{ "latency" : 654, "province" : "新疆", "timestamp" : "2016-10-28" }

{ "index": {}}

{ "latency" : 389, "province" : "新疆", "timestamp" : "2016-10-28" }

{ "index": {}}

{ "latency" : 302, "province" : "新疆", "timestamp" : "2016-10-29" }

 
 

二、pencentiles操作

1、查找tp50、tp90、tp99

 
 

GET /website/logs/_search

{

"size": 0,

"aggs": {

"latency_percentiles": {"percentiles": {"field": "latency","percents": [50,90,99]}},

"latency_late":{"avg": {"field": "latency"}}

}

}

 
 

执行结果如下：

 
 

"aggregations": {

"latency_late": {

"value": 201.91666666666666

},

"latency_percentiles": {

"values": {

"50.0": 108.5,

"90.0": 380.3,

"99.0": 624.8500000000001

}

}

}

}

注意是的，这个tp50等，均不是求里面的最大值，es经过了计算，但是这个计算到是是怎么个计算，我现在也还不知。

 
 

2、查看各省的情况

确定是那些省份比较慢

 
 

GET /website/logs/_search

{

"size": 0,

"aggs": {"group_by_province":{

"terms": {"field": "province"},

"aggs": {

"latency_percentiles": {"percentiles": {"field": "latency","percents": [50,90,99]}},

"latency_late":{"avg": {"field": "latency"}}

}

}

}

}

 
 

{

"aggregations": {

"group_by_province": {

"doc_count_error_upper_bound": 0,

"sum_other_doc_count": 0,

"buckets": [

{

"key": "新疆",

"doc_count": 6,

"latency_late": {

"value": 314.5

},

"latency_percentiles": {

"values": {

"50.0": 288.5,

"90.0": 521.5,

"99.0": 640.75

}

}

},

{

"key": "江苏",

"doc_count": 6,

"latency_late": {

"value": 89.33333333333333

},

"latency_percentiles": {

"values": {

"50.0": 87.5,

"90.0": 108.5,

"99.0": 111.65

}

}

}

]

}

}

}

可以看出新僵的网比较慢，所以要对新疆作处理。