日韩性视频-久久久蜜桃-www中文字幕-在线中文字幕av-亚洲欧美一区二区三区四区-撸久久-香蕉视频一区-久久无码精品丰满人妻-国产高潮av-激情福利社-日韩av网址大全-国产精品久久999-日本五十路在线-性欧美在线-久久99精品波多结衣一区-男女午夜免费视频-黑人极品ⅴideos精品欧美棵-人人妻人人澡人人爽精品欧美一区-日韩一区在线看-欧美a级在线免费观看

歡迎訪問 生活随笔!

生活随笔

當前位置: 首頁 > 编程资源 > 编程问答 >内容正文

编程问答

03.elasticsearch pipeline aggregation查询

發布時間:2024/2/28 编程问答 38 豆豆
生活随笔 收集整理的這篇文章主要介紹了 03.elasticsearch pipeline aggregation查询 小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

文章目錄

    • 1. pipeline aggregation查詢語法
      • 1. 符號代表
      • 2. 聚合層級
    • 2. pipeline aggregation 查詢類型概覽
      • 1. sibling aggregation
      • 2. parent aggregation
    • 3. 數據準備
    • 4.使用樣例
      • 1. sibling aggregation
        • 1. Avg Bucket Aggregation: sibling agg, 對bucket的統計值求average
          • 1. 普通metric求average
          • 2. 對特殊metric _count求avg
        • 2. Max Bucket Aggregation: sibling agg, 求bucket中的最大的bucket
        • 3. Min Bucket Aggregation: sibling agg, 求一組bucket中的最小的bucket
        • 4. Sum Bucket Aggregation: sibling agg, 對一組bucket求sum
        • 5. Stats Bucket Aggregation: sibling agg, 對一組bucket求stats
        • 6. Extended Stats Bucket Aggregation: sibling agg, 對一組bucket求extend stats
        • 7. Percentiles Bucket Aggregation: sibling agg, 對一組bucket求percentiles
      • 2. parent aggregation
        • 1. Derivative Aggregation: parent agg , 對histogram或date_histogram類型求導
        • 2. Moving Average Aggregation: parent agg, 對一組bucket求移動平均值
        • 3. Moving Function Aggregation: parent agg, 對一組bucket移動使用function
        • 4. Cumulative Sum Aggregation
        • 5. Bucket Script Aggregation: parent agg , 桶腳本聚合——基于父聚合的【一個或多個權值】,對這些權值通過腳本進行運算
        • 6. Bucket Selector Aggregation: parent agg , 對一組bucket執行過濾操作,只有滿足過濾條件的bucket會被保留到結果集當中
        • 7. Bucket Sort Aggregation:
        • 8. Serial Differencing Aggregation: parent agg 串行差分聚合

elasticsearch的aggregate查詢現在越來越豐富了,目前總共有4類。

  • metric aggregation: 主要是min,max,avg,sum,percetile 等單個統計指標的查詢
  • bucket aggregation: 主要是類似group by的查詢操作
  • matrix aggregation: 使用多個字段的值進行計算從而產生一個多維矩陣
  • pipeline aggregation: 主要是能夠在其他的aggregation進行一些附加的處理來增強數據
  • 本篇就主要學習pipeline aggregation

    1. pipeline aggregation查詢語法

    1. 符號代表

  • 聚合分隔符 >,指定父子聚合關系,如:“my_bucket>my_stats.avg”
  • 統計指標分隔符 .,指定聚合的特定統計指標
  • 聚合名稱 <name of the aggregation>,直接指定聚合的名稱
  • 統計指標 <name of the metric>,直接指定統計指標
  • 完整路徑 agg_name[> agg_name]*[. metrics],綜合利用上面的方式指定完整路徑
  • 特殊值 _count,bucket的文檔個數這個是一個特殊的統計指標(metric),可以在pipeline中對應bucket的doc數量。
  • 2. 聚合層級

    ** 1.parent **
    此類聚合的"輸入"是其【父聚合】的輸出,并對其進行進一步處理。一般不生成新的桶,而是對父聚合桶信息的增強,可以在parent agg 的每一個bucket中添加新的統計指標。
    這種典型的就是移動平均的計算,倒數計算,在parent中的每個bucket中都會增加一個統計指標。

    ** 2.sibling **
    此類聚合的輸入是其【兄弟聚合】的輸出。并能在同級上計算新的聚合bucket,也就會產生新的agg bucket 分組。
    這種典型的就是min,max等在原有bucket的基礎上再增加一個新的bucket來輸出min,max的值

    2. pipeline aggregation 查詢類型概覽

    1. sibling aggregation

  • Avg Bucket Aggregation: sibling agg, 對bucket的統計值求average
  • Max Bucket Aggregation: sibling agg, 求bucket中的最大的bucket
  • Min Bucket Aggregation: sibling agg, 求一組bucket中的最小的bucket
  • Sum Bucket Aggregation: sibling agg, 對一組bucket求sum
  • Stats Bucket Aggregation: sibling agg, 對一組bucket求stats
  • Extended Stats Bucket Aggregation: sibling agg, 對一組bucket求extend stats
  • Percentiles Bucket Aggregation: sibling agg, 對一組bucket求percentiles
  • 2. parent aggregation

  • Derivative Aggregation: parent agg , 對histogram或date_histogram類型求導
  • Moving Average Aggregation: parent agg, 對一組bucket求移動平均值,過期了
  • Moving Function Aggregation: parent agg, 最一組bucket移動使用function
  • Cumulative Sum Aggregation: 截止到當前bucket的累計求和
  • Bucket Script Aggregation: parent agg , 桶腳本聚合——基于父聚合的【一個或多個權值】,對這些權值通過腳本進行運算
  • Bucket Selector Aggregation: parent agg , 對一組bucket執行過濾操作,只有滿足過濾條件的bucket會被保留到結果集當中
  • Bucket Sort Aggregation: 對bucket進行排序
  • Serial Differencing Aggregation: parent agg 串行差分聚合
  • 3. 數據準備

    traffic_stats存儲的是博客每天的閱讀信息,包括閱讀量和最大閱讀耗時

    PUT traffic_stats {"mappings": {"properties": {"date": {"type": "date","format": "dateOptionalTime"},"visits": {"type": "integer"},"max_time_spent": {"type": "integer"}}} }

    數據

    PUT _bulk {"index":{"_index":"traffic_stats"}} {"visits":"488", "date":"2018-10-1", "max_time_spent":"900"} {"index":{"_index":"traffic_stats"}} {"visits":"783", "date":"2018-10-6", "max_time_spent":"928"} {"index":{"_index":"traffic_stats"}} {"visits":"789", "date":"2018-10-12", "max_time_spent":"1834"} {"index":{"_index":"traffic_stats"}} {"visits":"1299", "date":"2018-11-3", "max_time_spent":"592"} {"index":{"_index":"traffic_stats"}} {"visits":"394", "date":"2018-11-6", "max_time_spent":"1249"} {"index":{"_index":"traffic_stats"}} {"visits":"448", "date":"2018-11-24", "max_time_spent":"874"} {"index":{"_index":"traffic_stats"}} {"visits":"768", "date":"2018-12-18", "max_time_spent":"876"} {"index":{"_index":"traffic_stats"}} {"visits":"1194", "date":"2018-12-24", "max_time_spent":"1249"} {"index":{"_index":"traffic_stats"}} {"visits":"987", "date":"2018-12-28", "max_time_spent":"1599"} {"index":{"_index":"traffic_stats"}} {"visits":"872", "date":"2019-01-1", "max_time_spent":"828"} {"index":{"_index":"traffic_stats"}} {"visits":"972", "date":"2019-01-5", "max_time_spent":"723"} {"index":{"_index":"traffic_stats"}} {"visits":"827", "date":"2019-02-5", "max_time_spent":"1300"} {"index":{"_index":"traffic_stats"}} {"visits":"1584", "date":"2019-02-15", "max_time_spent":"1500"} {"index":{"_index":"traffic_stats"}} {"visits":"1604", "date":"2019-03-2", "max_time_spent":"1488"} {"index":{"_index":"traffic_stats"}} {"visits":"1499", "date":"2019-03-27", "max_time_spent":"1399"} {"index":{"_index":"traffic_stats"}} {"visits":"1392", "date":"2019-04-8", "max_time_spent":"1294"} {"index":{"_index":"traffic_stats"}} {"visits":"1247", "date":"2019-04-15", "max_time_spent":"1194"} {"index":{"_index":"traffic_stats"}} {"visits":"984", "date":"2019-05-15", "max_time_spent":"1184"} {"index":{"_index":"traffic_stats"}} {"visits":"1228", "date":"2019-05-18", "max_time_spent":"1485"} {"index":{"_index":"traffic_stats"}} {"visits":"1423", "date":"2019-06-14", "max_time_spent":"1452"} {"index":{"_index":"traffic_stats"}} {"visits":"1238", "date":"2019-06-24", "max_time_spent":"1329"} {"index":{"_index":"traffic_stats"}} {"visits":"1388", "date":"2019-07-14", "max_time_spent":"1542"} {"index":{"_index":"traffic_stats"}} {"visits":"1499", "date":"2019-07-24", "max_time_spent":"1742"} {"index":{"_index":"traffic_stats"}} {"visits":"1523", "date":"2019-08-13", "max_time_spent":"1552"} {"index":{"_index":"traffic_stats"}} {"visits":"1443", "date":"2019-08-19", "max_time_spent":"1511"} {"index":{"_index":"traffic_stats"}} {"visits":"1587", "date":"2019-09-14", "max_time_spent":"1497"} {"index":{"_index":"traffic_stats"}} {"visits":"1534", "date":"2019-09-27", "max_time_spent":"1434"}

    4.使用樣例

    1. sibling aggregation

    1. Avg Bucket Aggregation: sibling agg, 對bucket的統計值求average

    1. 普通metric求average

    1.先用date_histogram算一下每月有多少天有人閱讀和當月中閱讀量最多的一天對應的閱讀量

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0}},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3,"max_view_count" : {"value" : 1194.0}},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2,"max_view_count" : {"value" : 972.0}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0}},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0}},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2,"max_view_count" : {"value" : 1392.0}},{"key_as_string" : "2019-05-01T00:00:00.000Z","key" : 1556668800000,"doc_count" : 2,"max_view_count" : {"value" : 1228.0}},{"key_as_string" : "2019-06-01T00:00:00.000Z","key" : 1559347200000,"doc_count" : 2,"max_view_count" : {"value" : 1423.0}},{"key_as_string" : "2019-07-01T00:00:00.000Z","key" : 1561939200000,"doc_count" : 2,"max_view_count" : {"value" : 1499.0}},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]}}

    增加一個求average的sibling agg, 求每個月的閱讀量最多的一天的數平均值(每個月取瀏覽量最多的一天)

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"average_month_max": {"avg_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

    注意這里的bucket_path

    "buckets_path": "month_term>max_view_count.value"

    month_term和max_view_count都是agg name所以使用>來進行連接
    value是max_view_count的統計值,所以使用 .來進行連接

    生成的結果是

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},......{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"average_month_max" : {"value" : 1341.1666666666667}}

    注意下面這個運行的是沒有正確結果的
    勘誤,這里的使用方式有問題,這個可以改進的

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"}},"avg_month_max": {"avg_bucket": {"buckets_path": "month_term.doc_count" # 這個地方改成month_term._count 就會有結果了。}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3},......{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2}]},"avg_month_max" : {"value" : null # 這里面沒有正確返回}}

    因為他要求sibling agg是一個多個bucket的agg,而且對應的metric是一個數值型的,這里的month_term返回的是一個對象,可能就是這個原因

    這里需要勘誤一下,這個地方之所以不行是因為使用有誤,這里應該使用date_histogram返回的bucket的特殊metric _count。

    2. 對特殊metric _count求avg

    求每個月有閱讀記錄的天數,并給出天數最多的月份和每個月的平均閱讀天數

    GET traffic_stats/_search {"size": 0,"aggs": {"month_days_count": { # 有閱讀記錄的天數 ,每個bucket的doc_count,對應的metric為_count"date_histogram": {"field": "date","calendar_interval": "month"}},"max_day_month":{ # 閱讀天數最多的月份,這里使用了特殊metric _count"max_bucket": {"buckets_path": "month_days_count._count"}},"avg_day_each_month":{ # 每個月份的平均閱讀天數,這里使用了特殊metric _count"max_bucket": {"avg_bucket": {"buckets_path": "month_days_count._count"}}} }返回"aggregations" : {"month_days_count" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2},{"key_as_string" : "2019-05-01T00:00:00.000Z","key" : 1556668800000,"doc_count" : 2},{"key_as_string" : "2019-06-01T00:00:00.000Z","key" : 1559347200000,"doc_count" : 2},{"key_as_string" : "2019-07-01T00:00:00.000Z","key" : 1561939200000,"doc_count" : 2},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2}]},"max_day_month" : {"value" : 3.0,"keys" : ["2018-10-01T00:00:00.000Z","2018-11-01T00:00:00.000Z","2018-12-01T00:00:00.000Z"]},"avg_day_each_month" : {"value" : 2.25}}

    2. Max Bucket Aggregation: sibling agg, 求bucket中的最大的bucket

    承接average查詢,和avg_bucket類似

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"max_month_max": {"max_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }"aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"max_month_max" : {"value" : 1604.0,"keys" : ["2019-03-01T00:00:00.000Z"]}}

    3. Min Bucket Aggregation: sibling agg, 求一組bucket中的最小的bucket

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"min_month_max": {"min_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"min_month_max" : {"value" : 789.0,"keys" : ["2018-10-01T00:00:00.000Z"]}}

    4. Sum Bucket Aggregation: sibling agg, 對一組bucket求sum

    使用樣例

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"sum_month_max": {"sum_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"sum_month_max" : {"value" : 16094.0}}

    5. Stats Bucket Aggregation: sibling agg, 對一組bucket求stats

    使用樣例

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"stats_month_max": {"stats_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"stats_month_max" : {"count" : 12,"min" : 789.0,"max" : 1604.0,"avg" : 1341.1666666666667,"sum" : 16094.0}}

    6. Extended Stats Bucket Aggregation: sibling agg, 對一組bucket求extend stats

    使用樣例

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"extend_stats_month_max": {"extended_stats_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

    返回結果

    "aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"extend_stats_month_max" : {"count" : 12,"min" : 789.0,"max" : 1604.0,"avg" : 1341.1666666666667,"sum" : 16094.0,"sum_of_squares" : 2.231789E7,"variance" : 61096.13888888899,"std_deviation" : 247.17633157098393,"std_deviation_bounds" : {"upper" : 1835.5193298086347,"lower" : 846.8140035246988}}}

    7. Percentiles Bucket Aggregation: sibling agg, 對一組bucket求percentiles

    使用樣例

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}}}},"percentile_month_max": {"percentiles_bucket": {"buckets_path": "month_term>max_view_count.value"}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [......{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]},"percentile_month_max" : {"values" : {"1.0" : 789.0,"5.0" : 972.0,"25.0" : 1228.0,"50.0" : 1423.0,"75.0" : 1523.0,"95.0" : 1587.0,"99.0" : 1604.0}}}

    2. parent aggregation

    1. Derivative Aggregation: parent agg , 對histogram或date_histogram類型求導

    查詢樣例

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"deriv_month_max": { # 注意這里的層級變了,原來的sibling查詢是和month_term同級的"derivative": {"buckets_path": "max_view_count.value"}}}}} }

    求一階導數就是相鄰的差值,注意看上面的deriv_month_max 的層級變了

    返回

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0},"deriv_month_max" : {"value" : 510.0}},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3,"max_view_count" : {"value" : 1194.0},"deriv_month_max" : {"value" : -105.0}},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2,"max_view_count" : {"value" : 972.0},"deriv_month_max" : {"value" : -222.0}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0},"deriv_month_max" : {"value" : 612.0}},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0},"deriv_month_max" : {"value" : 20.0}},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2,"max_view_count" : {"value" : 1392.0},"deriv_month_max" : {"value" : -212.0}},{"key_as_string" : "2019-05-01T00:00:00.000Z","key" : 1556668800000,"doc_count" : 2,"max_view_count" : {"value" : 1228.0},"deriv_month_max" : {"value" : -164.0}},{"key_as_string" : "2019-06-01T00:00:00.000Z","key" : 1559347200000,"doc_count" : 2,"max_view_count" : {"value" : 1423.0},"deriv_month_max" : {"value" : 195.0}},{"key_as_string" : "2019-07-01T00:00:00.000Z","key" : 1561939200000,"doc_count" : 2,"max_view_count" : {"value" : 1499.0},"deriv_month_max" : {"value" : 76.0}},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0},"deriv_month_max" : {"value" : 24.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0},"deriv_month_max" : {"value" : 64.0}}]}}

    2. Moving Average Aggregation: parent agg, 對一組bucket求移動平均值

    這個現在過期了,當前推薦使用的是Moving Function Aggregation
    可以使用MovingFunctions.unweightedAvg(values) 來代替這個agg操作

    3. Moving Function Aggregation: parent agg, 對一組bucket移動使用function

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"move_avg_view": {"moving_fn": {"buckets_path": "max_view_count","window": 2,"script": "MovingFunctions.unweightedAvg(values)"}}}}} }

    這里窗口設置的為2,也就是臨近的兩個bucket求平均值,
    第一個bucket因為沒有其他bucket可以和他求平均,所以是null, 第二個bucket的均值等于第一個的,第三個bucket的移動均值是(bucket01+bucket02)/2
    返回結果

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0},"move_avg_view" : {"value" : null}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0},"move_avg_view" : {"value" : 789.0}},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3,"max_view_count" : {"value" : 1194.0},"move_avg_view" : {"value" : 1044.0}},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2,"max_view_count" : {"value" : 972.0},"move_avg_view" : {"value" : 1246.5}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0},"move_avg_view" : {"value" : 1083.0}},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0},"move_avg_view" : {"value" : 1278.0}},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2,"max_view_count" : {"value" : 1392.0},"move_avg_view" : {"value" : 1594.0}},{"key_as_string" : "2019-05-01T00:00:00.000Z","key" : 1556668800000,"doc_count" : 2,"max_view_count" : {"value" : 1228.0},"move_avg_view" : {"value" : 1498.0}},]}}

    4. Cumulative Sum Aggregation

    parent agg,
    累計和聚合——基于父聚合(只能是histogram或date_histogram類型)的某個權值,對權值在每一個桶中求所有之前的桶的該值累計的和。
    截止到當前bucket的累計統計值

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"cur_sum_view": {"cumulative_sum": {"buckets_path": "max_view_count"}}}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0},"cur_sum_view" : {"value" : 789.0}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0},"cur_sum_view" : {"value" : 2088.0}}......]}} }

    5. Bucket Script Aggregation: parent agg , 桶腳本聚合——基于父聚合的【一個或多個權值】,對這些權值通過腳本進行運算

    返回

    6. Bucket Selector Aggregation: parent agg , 對一組bucket執行過濾操作,只有滿足過濾條件的bucket會被保留到結果集當中

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"select_bucket": {"bucket_selector": {"buckets_path": {"var01": "max_view_count"},"script": "params.var01>1500"}}}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0}},{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0}},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}}]}}

    7. Bucket Sort Aggregation:

    parent agg, 對一組bucket進行排序z
    使用樣例

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"select_bucket": {"bucket_sort": {"sort": [{"max_view_count": {"order": "desc"}}]}}}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2019-03-01T00:00:00.000Z","key" : 1551398400000,"doc_count" : 2,"max_view_count" : {"value" : 1604.0}},{"key_as_string" : "2019-09-01T00:00:00.000Z","key" : 1567296000000,"doc_count" : 2,"max_view_count" : {"value" : 1587.0}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0}},{"key_as_string" : "2019-08-01T00:00:00.000Z","key" : 1564617600000,"doc_count" : 2,"max_view_count" : {"value" : 1523.0}},{"key_as_string" : "2019-07-01T00:00:00.000Z","key" : 1561939200000,"doc_count" : 2,"max_view_count" : {"value" : 1499.0}},{"key_as_string" : "2019-06-01T00:00:00.000Z","key" : 1559347200000,"doc_count" : 2,"max_view_count" : {"value" : 1423.0}},{"key_as_string" : "2019-04-01T00:00:00.000Z","key" : 1554076800000,"doc_count" : 2,"max_view_count" : {"value" : 1392.0}}]}}

    8. Serial Differencing Aggregation: parent agg 串行差分聚合

    可以配置的參數

    lag:滯后間隔(比如lag=7,表示每次從當前桶的值中減去其前面第7個桶的值)
    buckets_path:用于計算均值的權值路徑
    gap_policy:空桶處理策略(skip/insert_zeros)
    format:該聚合的輸出格式定義

    GET traffic_stats/_search {"size": 0,"aggs": {"month_term": {"date_histogram": {"field": "date","calendar_interval": "month"},"aggs": {"max_view_count": {"max": {"field": "visits"}},"diff_bucket": {"serial_diff": {"buckets_path": "max_view_count","lag": 2}}}}} }

    返回

    "aggregations" : {"month_term" : {"buckets" : [{"key_as_string" : "2018-10-01T00:00:00.000Z","key" : 1538352000000,"doc_count" : 3,"max_view_count" : {"value" : 789.0}},{"key_as_string" : "2018-11-01T00:00:00.000Z","key" : 1541030400000,"doc_count" : 3,"max_view_count" : {"value" : 1299.0}},{"key_as_string" : "2018-12-01T00:00:00.000Z","key" : 1543622400000,"doc_count" : 3,"max_view_count" : {"value" : 1194.0},"diff_bucket" : {"value" : 405.0}},{"key_as_string" : "2019-01-01T00:00:00.000Z","key" : 1546300800000,"doc_count" : 2,"max_view_count" : {"value" : 972.0},"diff_bucket" : {"value" : -327.0}},{"key_as_string" : "2019-02-01T00:00:00.000Z","key" : 1548979200000,"doc_count" : 2,"max_view_count" : {"value" : 1584.0},"diff_bucket" : {"value" : 390.0}},......]}}

    可以看到從第3個開始diff_bucket才開始有值,diff_bucket=(第3個bucket的max_view_count)-(第1個bucket的max_view_count)

    總結

    以上是生活随笔為你收集整理的03.elasticsearch pipeline aggregation查询的全部內容,希望文章能夠幫你解決所遇到的問題。

    如果覺得生活随笔網站內容還不錯,歡迎將生活随笔推薦給好友。