文章目錄

• • 模擬測(cè)試
  • • 測(cè)試
    • 返回結(jié)果
  • 實(shí)際應(yīng)用
  • • 創(chuàng)建Pipeline
    • 查看創(chuàng)建Pipeline
    • 新增數(shù)據(jù)測(cè)試
    • 查看新增數(shù)據(jù)
    • 創(chuàng)建索引時(shí)直接設(shè)置Pipeline

模擬測(cè)試

測(cè)試

POST _ingest/pipeline/_simulate {"pipeline": {"processors": [{"set": {"field": "timestamp","value": "{{_ingest.timestamp}}"}},{"script": {"lang": "painless","source": """ ZonedDateTime zdt = ZonedDateTime.parse(ctx.timestamp); DateTimeFormatter dtf = DateTimeFormatter.ofPattern( "yyyy-MM-dd HH:mm:ss"); String datetime = zdt.format(dtf); ctx.newdate = datetime; ctx.newdate2 = System.currentTimeMillis()/1000; ctx.newdate3 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());"""}}]},"docs": [{"_source": {"message": "測(cè)試"}}] }

注意：這里可能有時(shí)區(qū)問(wèn)題，慢8個(gè)小時(shí)，可臨時(shí)使用 ctx.date_zsh = new Date(System.currentTimeMillis()+1000l*60*60*8); 來(lái)處理

我們使用_ingest.timestamp 與painless 多種方式設(shè)置了數(shù)據(jù)最新更新時(shí)間

返回結(jié)果

newdate2 為數(shù)據(jù)更新時(shí)間秒，newdate為格式轉(zhuǎn)換后的數(shù)據(jù)，timestamp 為 _ingest.timestamp 獲取到的時(shí)間

{"docs" : [{"doc" : {"_index" : "_index","_type" : "_doc","_id" : "_id","_source" : {"newdate2" : 1624848304,"message" : "測(cè)試","newdate" : "2021-06-28 02:45:04","timestamp" : "2021-06-28T02:45:04.759053131Z"},"_ingest" : {"timestamp" : "2021-06-28T02:45:04.759053131Z"}}}] }

實(shí)際應(yīng)用

創(chuàng)建Pipeline

PUT _ingest/pipeline/add_timestamp {"processors": [{"set": {"field": "timestamp","value": "{{_ingest.timestamp}}"}},{"script": {"lang": "painless","source": """ ZonedDateTime zdt = ZonedDateTime.parse(ctx.timestamp); DateTimeFormatter dtf = DateTimeFormatter.ofPattern( "yyyy-MM-dd HH:mm:ss"); String datetime = zdt.format(dtf); ctx.newdate = datetime; ctx.newdate2 = System.currentTimeMillis()/1000; """}}]}

查看創(chuàng)建Pipeline

GET _ingest/pipeline/add_timestamp{"add_timestamp" : {"processors" : [{"set" : {"field" : "timestamp","value" : "{{_ingest.timestamp}}"}},{"script" : {"lang" : "painless","source" : """ ZonedDateTime zdt = ZonedDateTime.parse(ctx.timestamp); DateTimeFormatter dtf = DateTimeFormatter.ofPattern( "yyyy-MM-dd HH:mm:ss"); String datetime = zdt.format(dtf); ctx.newdate = datetime; ctx.newdate2 = System.currentTimeMillis()/1000; """}}]} }

新增數(shù)據(jù)測(cè)試

PUT test_index_20210628/_doc/1?pipeline=add_timestamp {"test":"測(cè)試數(shù)據(jù)" }

查看新增數(shù)據(jù)

GET test_index_20210628/_doc/1 {"_index" : "test_index_20210628","_type" : "_doc","_id" : "1","_version" : 1,"_seq_no" : 0,"_primary_term" : 1,"found" : true,"_source" : {"newdate2" : 1624849340,"test" : "測(cè)試數(shù)據(jù)","newdate" : "2021-06-28 03:02:20","timestamp" : "2021-06-28T03:02:20.252887295Z"} }

創(chuàng)建索引時(shí)直接設(shè)置Pipeline

我們也可以在創(chuàng)建索引時(shí)設(shè)置Pipeline，這時(shí)就不需要每次添加數(shù)據(jù)時(shí)指定Pipeline

# 創(chuàng)建索引指定pipeline PUT test_index_20210628_02 {"settings": {"default_pipeline": "add_timestamp"} }# 添加測(cè)試數(shù)據(jù) PUT test_index_20210628_02/_doc/1 {"test":"測(cè)試數(shù)據(jù)" }# 獲取數(shù)據(jù) GET test_index_20210628_02/_doc/1# 返回結(jié)果 {"_index" : "test_index_20210628_02","_type" : "_doc","_id" : "1","_version" : 1,"_seq_no" : 0,"_primary_term" : 1,"found" : true,"_source" : {"newdate2" : 1624849478,"test" : "測(cè)試數(shù)據(jù)","newdate" : "2021-06-28 03:04:38","timestamp" : "2021-06-28T03:04:38.940542643Z"} }

個(gè)人公眾號(hào)(大數(shù)據(jù)學(xué)習(xí)交流): hadoopwiki

總結(jié)

以上是生活随笔為你收集整理的ElasticSearch Pipeline 为新增数据设置更新时间的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。