文章目錄

• • 1. pipeline aggregation查詢語法
  • • 1. 符號(hào)代表
    • 2. 聚合層級(jí)
  • 2. pipeline aggregation 查詢類型概覽
  • • 1. sibling aggregation
    • 2. parent aggregation
  • 3. 數(shù)據(jù)準(zhǔn)備
  • 4.使用樣例
  • • 1. sibling aggregation
    • • 1. Avg Bucket Aggregation: sibling agg, 對(duì)bucket的統(tǒng)計(jì)值求average
      • • 1. 普通metric求average
        • 2. 對(duì)特殊metric _count求avg
      • 2. Max Bucket Aggregation: sibling agg, 求bucket中的最大的bucket
      • 3. Min Bucket Aggregation: sibling agg, 求一組bucket中的最小的bucket
      • 4. Sum Bucket Aggregation: sibling agg, 對(duì)一組bucket求sum
      • 5. Stats Bucket Aggregation: sibling agg, 對(duì)一組bucket求stats
      • 6. Extended Stats Bucket Aggregation: sibling agg, 對(duì)一組bucket求extend stats
      • 7. Percentiles Bucket Aggregation: sibling agg, 對(duì)一組bucket求percentiles
    • 2. parent aggregation
    • • 1. Derivative Aggregation: parent agg , 對(duì)histogram或date_histogram類型求導(dǎo)
      • 2. Moving Average Aggregation: parent agg, 對(duì)一組bucket求移動(dòng)平均值
      • 3. Moving Function Aggregation: parent agg, 對(duì)一組bucket移動(dòng)使用function
      • 4. Cumulative Sum Aggregation
      • 5. Bucket Script Aggregation: parent agg , 桶腳本聚合——基于父聚合的【一個(gè)或多個(gè)權(quán)值】，對(duì)這些權(quán)值通過腳本進(jìn)行運(yùn)算
      • 6. Bucket Selector Aggregation: parent agg , 對(duì)一組bucket執(zhí)行過濾操作，只有滿足過濾條件的bucket會(huì)被保留到結(jié)果集當(dāng)中
      • 7. Bucket Sort Aggregation:
      • 8. Serial Differencing Aggregation: parent agg 串行差分聚合

elasticsearch的aggregate查詢現(xiàn)在越來越豐富了，目前總共有4類。

metric aggregation: 主要是min,max,avg,sum,percetile 等單個(gè)統(tǒng)計(jì)指標(biāo)的查詢

bucket aggregation: 主要是類似group by的查詢操作

matrix aggregation: 使用多個(gè)字段的值進(jìn)行計(jì)算從而產(chǎn)生一個(gè)多維矩陣

pipeline aggregation: 主要是能夠在其他的aggregation進(jìn)行一些附加的處理來增強(qiáng)數(shù)據(jù)

本篇就主要學(xué)習(xí)pipeline aggregation

1. pipeline aggregation查詢語法

1. 符號(hào)代表

聚合分隔符 >，指定父子聚合關(guān)系，如：“my_bucket>my_stats.avg”

統(tǒng)計(jì)指標(biāo)分隔符 .，指定聚合的特定統(tǒng)計(jì)指標(biāo)

聚合名稱 <name of the aggregation>，直接指定聚合的名稱

統(tǒng)計(jì)指標(biāo) <name of the metric>，直接指定統(tǒng)計(jì)指標(biāo)

完整路徑 agg_name[> agg_name]*[. metrics]，綜合利用上面的方式指定完整路徑

特殊值 _count，bucket的文檔個(gè)數(shù)這個(gè)是一個(gè)特殊的統(tǒng)計(jì)指標(biāo)（metric），可以在pipeline中對(duì)應(yīng)bucket的doc數(shù)量。

2. 聚合層級(jí)

** 1.parent **
此類聚合的"輸入"是其【父聚合】的輸出，并對(duì)其進(jìn)行進(jìn)一步處理。一般不生成新的桶，而是對(duì)父聚合桶信息的增強(qiáng),可以在parent agg 的每一個(gè)bucket中添加新的統(tǒng)計(jì)指標(biāo)。
這種典型的就是移動(dòng)平均的計(jì)算，倒數(shù)計(jì)算,在parent中的每個(gè)bucket中都會(huì)增加一個(gè)統(tǒng)計(jì)指標(biāo)。

** 2.sibling **
此類聚合的輸入是其【兄弟聚合】的輸出。并能在同級(jí)上計(jì)算新的聚合bucket,也就會(huì)產(chǎn)生新的agg bucket 分組。
這種典型的就是min,max等在原有bucket的基礎(chǔ)上再增加一個(gè)新的bucket來輸出min,max的值

2. pipeline aggregation 查詢類型概覽

1. sibling aggregation

Avg Bucket Aggregation: sibling agg, 對(duì)bucket的統(tǒng)計(jì)值求average

Max Bucket Aggregation: sibling agg, 求bucket中的最大的bucket

Min Bucket Aggregation: sibling agg, 求一組bucket中的最小的bucket

Sum Bucket Aggregation: sibling agg, 對(duì)一組bucket求sum

Stats Bucket Aggregation: sibling agg, 對(duì)一組bucket求stats

Extended Stats Bucket Aggregation: sibling agg, 對(duì)一組bucket求extend stats

Percentiles Bucket Aggregation: sibling agg, 對(duì)一組bucket求percentiles

2. parent aggregation

Derivative Aggregation: parent agg , 對(duì)histogram或date_histogram類型求導(dǎo)

Moving Average Aggregation: parent agg, 對(duì)一組bucket求移動(dòng)平均值,過期了

Moving Function Aggregation: parent agg, 最一組bucket移動(dòng)使用function

Cumulative Sum Aggregation: 截止到當(dāng)前bucket的累計(jì)求和

Bucket Script Aggregation: parent agg , 桶腳本聚合——基于父聚合的【一個(gè)或多個(gè)權(quán)值】，對(duì)這些權(quán)值通過腳本進(jìn)行運(yùn)算

Bucket Selector Aggregation: parent agg , 對(duì)一組bucket執(zhí)行過濾操作，只有滿足過濾條件的bucket會(huì)被保留到結(jié)果集當(dāng)中

Bucket Sort Aggregation: 對(duì)bucket進(jìn)行排序

Serial Differencing Aggregation: parent agg 串行差分聚合

3. 數(shù)據(jù)準(zhǔn)備

traffic_stats存儲(chǔ)的是博客每天的閱讀信息，包括閱讀量和最大閱讀耗時(shí)

PUT traffic_stats {"mappings": {"properties": {"date": {"type": "date","format": "dateOptionalTime"},"visits": {"type": "integer"},"max_time_spent": {"type": "integer"}}} }

數(shù)據(jù)

PUT _bulk {"index":{"_index":"traffic_stats"}} {"visits":"488", "date":"2018-10-1", "max_time_spent":"900"} {"index":{"_index":"traffic_stats"}} {"visits":"783", "date":"2018-10-6", "max_time_spent":"928"} {"index":{"_index":"traffic_stats"}} {"visits":"789", "date":"2018-10-12", "max_time_spent":"1834"} {"index":{"_index":"traffic_stats"}} {"visits":"1299", "date":"2018-11-3", "max_time_spent":"592"} {"index":{"_index":"traffic_stats"}} {"visits":"394", "date":"2018-11-6", "max_time_spent":"1249"} {"index":{"_index":"traffic_stats"}} {"visits":"448", "date":"2018-11-24", "max_time_spent":"874"} {"index":{"_index":"traffic_stats"}} {"visits":"768", "date":"2018-12-18", "max_time_spent":"876"} {"index":{"_index":"traffic_stats"}} {"visits":"1194", "date":"2018-12-24", "max_time_spent":"1249"} {"index":{"_index":"traffic_stats"}} {"visits":"987", "date":"2018-12-28", "max_time_spent":"1599"} {"index":{"_index":"traffic_stats"}} {"visits":"872", "date":"2019-01-1", "max_time_spent":"828"} {"index":{"_index":"traffic_stats"}} {"visits":"972", "date":"2019-01-5", "max_time_spent":"723"} {"index":{"_index":"traffic_stats"}} {"visits":"827", "date":"2019-02-5", "max_time_spent":"1300"} {"index":{"_index":"traffic_stats"}} {"visits":"1584", "date":"2019-02-15", "max_time_spent":"1500"} {"index":{"_index":"traffic_stats"}} {"visits":"1604", "date":"2019-03-2", "max_time_spent":"1488"} {"index":{"_index":"traffic_stats"}} {"visits":"1499", "date":"2019-03-27", "max_time_spent":"1399"} {"index":{"_index":"traffic_stats"}} {"visits":"1392", "date":"2019-04-8", "max_time_spent":"1294"} {"index":{"_index":"traffic_stats"}} {"visits":"1247", "date":"2019-04-15", "max_time_spent":"1194"} {"index":{"_index":"traffic_stats"}} {"visits":"984", "date":"2019-05-15", "max_time_spent":"1184"} {"index":{"_index":"traffic_stats"}} {"visits":"1228", "date":"2019-05-18", "max_time_spent":"1485"} {"index":{"_index":"traffic_stats"}} {"visits":"1423", "date":"2019-06-14", "max_time_spent":"1452"} {"index":{"_index":"traffic_stats"}} {"visits":"1238", "date":"2019-06-24", "max_time_spent":"1329"} {"index":{"_index":"traffic_stats"}} {"visits":"1388", "date":"2019-07-14", "max_time_spent":"1542"} {"index":{"_index":"traffic_stats"}} {"visits":"1499", "date":"2019-07-24", "max_time_spent":"1742"} {"index":{"_index":"traffic_stats"}} {"visits":"1523", "date":"2019-08-13", "max_time_spent":"1552"} {"index":{"_index":"traffic_stats"}} {"visits":"1443", "date":"2019-08-19", "max_time_spent":"1511"} {"index":{"_index":"traffic_stats"}} {"visits":"1587", "date":"2019-09-14", "max_time_spent":"1497"} {"index":{"_index":"traffic_stats"}} {"visits":"1534", "date":"2019-09-27", "max_time_spent":"1434"}

4.使用樣例

1. sibling aggregation

1. Avg Bucket Aggregation: sibling agg, 對(duì)bucket的統(tǒng)計(jì)值求average

1. 普通metric求average

1.先用date_histogram算一下每月有多少天有人閱讀和當(dāng)月中閱讀量最多的一天對(duì)應(yīng)的閱讀量

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0}},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3,"max_view_count" : {"value" : 1194.0}},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2,"max_view_count" : {"value" : 972.0}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0}},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0}},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2,"max_view_count" : {"value" : 1392.0}},{"key_as_string" : "2019-05-01T00:00:00.000Z","key" : 1556668800000,"doc_count" : 2,"max_view_count" : {"value" : 1228.0}},{"key_as_string" : "2019-06-01T00:00:00.000Z","key" : 1559347200000,"doc_count" : 2,"max_view_count" : {"value" : 1423.0}},{"key_as_string" : "2019-07-01T00:00:00.000Z","key" : 1561939200000,"doc_count" : 2,"max_view_count" : {"value" : 1499.0}},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]}}

增加一個(gè)求average的sibling agg, 求每個(gè)月的閱讀量最多的一天的數(shù)平均值（每個(gè)月取瀏覽量最多的一天）

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"average_month_max": {"avg_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

注意這里的bucket_path

"buckets_path": "month_term>max_view_count.value"

month_term和max_view_count都是agg name所以使用>來進(jìn)行連接
value是max_view_count的統(tǒng)計(jì)值，所以使用 .來進(jìn)行連接

生成的結(jié)果是

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},......{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"average_month_max" : {"value" : 1341.1666666666667}}

注意下面這個(gè)運(yùn)行的是沒有正確結(jié)果的
勘誤，這里的使用方式有問題，這個(gè)可以改進(jìn)的

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"}},"avg_month_max": {"avg_bucket": {"buckets_path": "month_term.doc_count" # 這個(gè)地方改成month_term._count 就會(huì)有結(jié)果了。}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3},......{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2}]},"avg_month_max" : {"value" : null # 這里面沒有正確返回}}

因?yàn)樗髎ibling agg是一個(gè)多個(gè)bucket的agg,而且對(duì)應(yīng)的metric是一個(gè)數(shù)值型的,這里的month_term返回的是一個(gè)對(duì)象，可能就是這個(gè)原因

這里需要勘誤一下，這個(gè)地方之所以不行是因?yàn)槭褂糜姓`，這里應(yīng)該使用date_histogram返回的bucket的特殊metric _count。

2. 對(duì)特殊metric _count求avg

求每個(gè)月有閱讀記錄的天數(shù)，并給出天數(shù)最多的月份和每個(gè)月的平均閱讀天數(shù)

GET traffic_stats/_search {"size": 0,"aggs": {"month_days_count": { # 有閱讀記錄的天數(shù) ，每個(gè)bucket的doc_count,對(duì)應(yīng)的metric為_count"date_histogram": {"field": "date","calendar_interval": "month"}},"max_day_month":{ # 閱讀天數(shù)最多的月份，這里使用了特殊metric _count"max_bucket": {"buckets_path": "month_days_count._count"}},"avg_day_each_month":{ # 每個(gè)月份的平均閱讀天數(shù)，這里使用了特殊metric _count"max_bucket": {"avg_bucket": {"buckets_path": "month_days_count._count"}}} }返回"aggregations" : {"month_days_count" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2},{"key_as_string" : "2019-05-01T00:00:00.000Z","key" : 1556668800000,"doc_count" : 2},{"key_as_string" : "2019-06-01T00:00:00.000Z","key" : 1559347200000,"doc_count" : 2},{"key_as_string" : "2019-07-01T00:00:00.000Z","key" : 1561939200000,"doc_count" : 2},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2}]},"max_day_month" : {"value" : 3.0,"keys" : ["2018-10-01T00:00:00.000Z","2018-11-01T00:00:00.000Z","2018-12-01T00:00:00.000Z"]},"avg_day_each_month" : {"value" : 2.25}}

2. Max Bucket Aggregation: sibling agg, 求bucket中的最大的bucket

承接average查詢，和avg_bucket類似

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"max_month_max": {"max_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }"aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"max_month_max" : {"value" : 1604.0,"keys" : ["2019-03-01T00:00:00.000Z"]}}

3. Min Bucket Aggregation: sibling agg, 求一組bucket中的最小的bucket

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"min_month_max": {"min_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"min_month_max" : {"value" : 789.0,"keys" : ["2018-10-01T00:00:00.000Z"]}}

4. Sum Bucket Aggregation: sibling agg, 對(duì)一組bucket求sum

使用樣例

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"sum_month_max": {"sum_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"sum_month_max" : {"value" : 16094.0}}

5. Stats Bucket Aggregation: sibling agg, 對(duì)一組bucket求stats

使用樣例

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"stats_month_max": {"stats_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"stats_month_max" : {"count" : 12,"min" : 789.0,"max" : 1604.0,"avg" : 1341.1666666666667,"sum" : 16094.0}}

6. Extended Stats Bucket Aggregation: sibling agg, 對(duì)一組bucket求extend stats

使用樣例

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"extend_stats_month_max": {"extended_stats_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

返回結(jié)果

"aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"extend_stats_month_max" : {"count" : 12,"min" : 789.0,"max" : 1604.0,"avg" : 1341.1666666666667,"sum" : 16094.0,"sum_of_squares" : 2.231789E7,"variance" : 61096.13888888899,"std_deviation" : 247.17633157098393,"std_deviation_bounds" : {"upper" : 1835.5193298086347,"lower" : 846.8140035246988}}}

7. Percentiles Bucket Aggregation: sibling agg, 對(duì)一組bucket求percentiles

使用樣例

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"percentile_month_max": {"percentiles_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"percentile_month_max" : {"values" : {"1.0" : 789.0,"5.0" : 972.0,"25.0" : 1228.0,"50.0" : 1423.0,"75.0" : 1523.0,"95.0" : 1587.0,"99.0" : 1604.0}}}

2. parent aggregation

1. Derivative Aggregation: parent agg , 對(duì)histogram或date_histogram類型求導(dǎo)

查詢樣例

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"deriv_month_max": { # 注意這里的層級(jí)變了，原來的sibling查詢是和month_term同級(jí)的"derivative": {"buckets_path": "max_view_count.value"}}}}} }

求一階導(dǎo)數(shù)就是相鄰的差值，注意看上面的deriv_month_max 的層級(jí)變了

返回

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0},"deriv_month_max" : {"value" : 510.0}},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3,"max_view_count" : {"value" : 1194.0},"deriv_month_max" : {"value" : -105.0}},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2,"max_view_count" : {"value" : 972.0},"deriv_month_max" : {"value" : -222.0}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0},"deriv_month_max" : {"value" : 612.0}},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0},"deriv_month_max" : {"value" : 20.0}},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2,"max_view_count" : {"value" : 1392.0},"deriv_month_max" : {"value" : -212.0}},{"key_as_string" : "2019-05-01T00:00:00.000Z","key" : 1556668800000,"doc_count" : 2,"max_view_count" : {"value" : 1228.0},"deriv_month_max" : {"value" : -164.0}},{"key_as_string" : "2019-06-01T00:00:00.000Z","key" : 1559347200000,"doc_count" : 2,"max_view_count" : {"value" : 1423.0},"deriv_month_max" : {"value" : 195.0}},{"key_as_string" : "2019-07-01T00:00:00.000Z","key" : 1561939200000,"doc_count" : 2,"max_view_count" : {"value" : 1499.0},"deriv_month_max" : {"value" : 76.0}},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0},"deriv_month_max" : {"value" : 24.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0},"deriv_month_max" : {"value" : 64.0}}]}}

2. Moving Average Aggregation: parent agg, 對(duì)一組bucket求移動(dòng)平均值

這個(gè)現(xiàn)在過期了，當(dāng)前推薦使用的是Moving Function Aggregation
可以使用MovingFunctions.unweightedAvg(values) 來代替這個(gè)agg操作

3. Moving Function Aggregation: parent agg, 對(duì)一組bucket移動(dòng)使用function

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"move_avg_view": {"moving_fn": {"buckets_path": "max_view_count","window": 2,"script": "MovingFunctions.unweightedAvg(values)"}}}}} }

這里窗口設(shè)置的為2，也就是臨近的兩個(gè)bucket求平均值，
第一個(gè)bucket因?yàn)闆]有其他bucket可以和他求平均，所以是null, 第二個(gè)bucket的均值等于第一個(gè)的，第三個(gè)bucket的移動(dòng)均值是(bucket01+bucket02)/2
返回結(jié)果

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0},"move_avg_view" : {"value" : null}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0},"move_avg_view" : {"value" : 789.0}},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3,"max_view_count" : {"value" : 1194.0},"move_avg_view" : {"value" : 1044.0}},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2,"max_view_count" : {"value" : 972.0},"move_avg_view" : {"value" : 1246.5}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0},"move_avg_view" : {"value" : 1083.0}},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0},"move_avg_view" : {"value" : 1278.0}},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2,"max_view_count" : {"value" : 1392.0},"move_avg_view" : {"value" : 1594.0}},{"key_as_string" : "2019-05-01T00:00:00.000Z","key" : 1556668800000,"doc_count" : 2,"max_view_count" : {"value" : 1228.0},"move_avg_view" : {"value" : 1498.0}},]}}

4. Cumulative Sum Aggregation

parent agg,
累計(jì)和聚合——基于父聚合（只能是histogram或date_histogram類型）的某個(gè)權(quán)值，對(duì)權(quán)值在每一個(gè)桶中求所有之前的桶的該值累計(jì)的和。
截止到當(dāng)前bucket的累計(jì)統(tǒng)計(jì)值

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"cur_sum_view": {"cumulative_sum": {"buckets_path": "max_view_count"}}}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0},"cur_sum_view" : {"value" : 789.0}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0},"cur_sum_view" : {"value" : 2088.0}}......]}} }

5. Bucket Script Aggregation: parent agg , 桶腳本聚合——基于父聚合的【一個(gè)或多個(gè)權(quán)值】，對(duì)這些權(quán)值通過腳本進(jìn)行運(yùn)算

返回

6. Bucket Selector Aggregation: parent agg , 對(duì)一組bucket執(zhí)行過濾操作，只有滿足過濾條件的bucket會(huì)被保留到結(jié)果集當(dāng)中

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"select_bucket": {"bucket_selector": {"buckets_path": {"var01": "max_view_count"},"script": "params.var01>1500"}}}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0}},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0}},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]}}

7. Bucket Sort Aggregation:

parent agg, 對(duì)一組bucket進(jìn)行排序z
使用樣例

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"select_bucket": {"bucket_sort": {"sort": [{"max_view_count": {"order": "desc"}}]}}}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0}},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-07-01T00:00:00.000Z","key" : 1561939200000,"doc_count" : 2,"max_view_count" : {"value" : 1499.0}},{"key_as_string" : "2019-06-01T00:00:00.000Z","key" : 1559347200000,"doc_count" : 2,"max_view_count" : {"value" : 1423.0}},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2,"max_view_count" : {"value" : 1392.0}}]}}

8. Serial Differencing Aggregation: parent agg 串行差分聚合

可以配置的參數(shù)

lag：滯后間隔（比如lag=7，表示每次從當(dāng)前桶的值中減去其前面第7個(gè)桶的值）
buckets_path：用于計(jì)算均值的權(quán)值路徑
gap_policy：空桶處理策略（skip/insert_zeros）
format：該聚合的輸出格式定義

GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"diff_bucket": {"serial_diff": {"buckets_path": "max_view_count","lag": 2}}}}} }

返回

"aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0}},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3,"max_view_count" : {"value" : 1194.0},"diff_bucket" : {"value" : 405.0}},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2,"max_view_count" : {"value" : 972.0},"diff_bucket" : {"value" : -327.0}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0},"diff_bucket" : {"value" : 390.0}},......]}}

可以看到從第3個(gè)開始diff_bucket才開始有值，diff_bucket=（第3個(gè)bucket的max_view_count）-（第1個(gè)bucket的max_view_count）

總結(jié)

以上是生活随笔為你收集整理的03.elasticsearch pipeline aggregation查询的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。