1． query DSL入門

1.1 DSL

query string 后邊的參數(shù)原來越多，搜索條件越來越復(fù)雜，不能滿足需求。

GET /book/_search?q=name:java&size=10&from=0&sort=price:desc

DSL:Domain Specified Language，特定領(lǐng)域的語(yǔ)言

es特有的搜索語(yǔ)言，可在請(qǐng)求體中攜帶搜索條件，功能強(qiáng)大。

查詢?nèi)?GET /book/_search

GET /book/_search {"query": { "match_all": {} } }

排序 GET /book/_search?sort=price:desc

GET /book/_search {"query" : {"match" : {"name" : " java"}},"sort": [{ "price": "desc" }] }

分頁(yè)查詢 GET /book/_search?size=10&from=0

GET /book/_search {"query": { "match_all": {} },"from": 0,"size": 1 }

指定返回字段 GET /book/ _search? _source=name,studymodel

GET /book/_search {"query": { "match_all": {} },"_source": ["name", "studymodel"] }

通過組合以上各種類型查詢，實(shí)現(xiàn)復(fù)雜查詢。

1.2． Query DSL語(yǔ)法

{QUERY_NAME: {ARGUMENT: VALUE,ARGUMENT: VALUE,...} } {QUERY_NAME: {FIELD_NAME: {ARGUMENT: VALUE,ARGUMENT: VALUE,...}} } GET /test_index/_search {"query": {"match": {"test_field": "test"}} }

1.3 組合多個(gè)搜索條件

搜索需求：title必須包含elasticsearch，content可以包含elasticsearch也可以不包含，author_id必須不為111

sql where and or !=

初始數(shù)據(jù)：

POST /website/_doc/1 {"title": "my hadoop article","content": "hadoop is very bad","author_id": 111 } ? POST /website/_doc/2 {"title": "my elasticsearch article","content": "es is very bad","author_id": 112 } POST /website/_doc/3 {"title": "my elasticsearch article","content": "es is very goods","author_id": 111 }

搜索：

GET /website/_doc/_search {"query": {"bool": {"must": [{"match": {"title": "elasticsearch"}}],"should": [{"match": {"content": "elasticsearch"}}],"must_not": [{"match": {"author_id": 111}}]}} }

返回：

{"took" : 488,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 0.47000363,"hits" : [{"_index" : "website","_type" : "_doc","_id" : "2","_score" : 0.47000363,"_source" : {"title" : "my elasticsearch article","content" : "es is very bad","author_id" : 112}}]} }

更復(fù)雜的搜索需求：

select * from test_index where name='tom' or (hired =true and (personality ='good' and rude != true ))

GET /test_index/_search {"query": {"bool": {"must": { "match":{ "name": "tom" }},"should": [{ "match":{ "hired": true }},{ "bool": {"must":{ "match": { "personality": "good" }},"must_not": { "match": { "rude": true }}}}],"minimum_should_match": 1}} }

2． full-text search 全文檢索

2.1 全文檢索

重新創(chuàng)建book索引

PUT /book/ {"settings": {"number_of_shards": 1,"number_of_replicas": 0},"mappings": {"properties": {"name":{"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},"description":{"type": "text","analyzer": "ik_max_word","search_analyzer": "ik_smart"},"studymodel":{"type": "keyword"},"price":{"type": "double"},"timestamp": {"type": "date","format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"},"pic":{"type":"text","index":false}}} }

插入數(shù)據(jù)

PUT /book/_doc/1 { "name": "Bootstrap開發(fā)", "description": "Bootstrap是由Twitter推出的一個(gè)前臺(tái)頁(yè)面開發(fā)css框架，是一個(gè)非常流行的開發(fā)框架，此框架集成了多種頁(yè)面效果。此開發(fā)框架包含了大量的CSS、JS程序代碼，可以幫助開發(fā)者（尤其是不擅長(zhǎng)css頁(yè)面開發(fā)的程序人員）輕松的實(shí)現(xiàn)一個(gè)css，不受瀏覽器限制的精美界面css效果。", "studymodel": "201002", "price":38.6, "timestamp":"2019-08-25 19:11:35", "pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg", "tags": [ "bootstrap", "dev"] } ? PUT /book/_doc/2 { "name": "java編程思想", "description": "java語(yǔ)言是世界第一編程語(yǔ)言，在軟件開發(fā)領(lǐng)域使用人數(shù)最多。", "studymodel": "201001", "price":68.6, "timestamp":"2019-08-25 19:11:35", "pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg", "tags": [ "java", "dev"] } ? PUT /book/_doc/3 { "name": "spring開發(fā)基礎(chǔ)", "description": "spring 在java領(lǐng)域非常流行，java程序員都在用。", "studymodel": "201001", "price":88.6, "timestamp":"2019-08-24 19:11:35", "pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg", "tags": [ "spring", "java"] }

搜索

GET /book/_search {"query" : {"match" : {"description" : "java程序員"}} }

2.2 _score初探

{"took" : 1,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 2,"relation" : "eq"},"max_score" : 2.137549,"hits" : [{"_index" : "book","_type" : "_doc","_id" : "3","_score" : 2.137549,"_source" : {"name" : "spring開發(fā)基礎(chǔ)","description" : "spring 在java領(lǐng)域非常流行，java程序員都在用。","studymodel" : "201001","price" : 88.6,"timestamp" : "2019-08-24 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["spring","java"]}},{"_index" : "book","_type" : "_doc","_id" : "2","_score" : 0.57961315,"_source" : {"name" : "java編程思想","description" : "java語(yǔ)言是世界第一編程語(yǔ)言，在軟件開發(fā)領(lǐng)域使用人數(shù)最多。","studymodel" : "201001","price" : 68.6,"timestamp" : "2019-08-25 19:11:35","pic" : "group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg","tags" : ["java","dev"]}}]} }

結(jié)果分析

1、建立索引時(shí), description字段 term倒排索引

java 2,3

程序員 3

2、搜索時(shí)，直接找description中含有java的文檔 2,3，并且3號(hào)文檔含有兩個(gè)java字段，一個(gè)程序員，所以得分高，排在前面。2號(hào)文檔含有一個(gè)java，排在后面。

3． DSL 語(yǔ)法練習(xí)

3.1 match_all

查詢所有文檔

GET /book/_search {"query": {"match_all": {}} }

3.2 match

match query 知道分詞器的存在，會(huì)對(duì)field進(jìn)行分詞操作，然后再查詢

GET /book/_search {"query": { "match": { "description": "java程序員"}} }

如果想不分詞直接查找，可以加.keyword

GET /book/_search {"query": { "match": { "description.keyword": "java程序員 西二旗"}} }

3.3 multi_match

可以指定多個(gè)字段

GET /book/_search {"query": {"multi_match": {"query": "java程序員","fields": ["name", "description"]}} }

3.4、range query

范圍查詢

GET /book/_search {"query": {"range": {"price": {"gte": 80,"lte": 90}}} }

3.5、term query

term query會(huì)去倒排索引中尋找確切的term，它并不知道分詞器的存在。這種查詢適合keyword 、numeric、date

GET /book/_search {"query": {"term": {"description": "java程序員"}} }

3.6、terms query

查詢某個(gè)字段里含有多個(gè)關(guān)鍵詞的文檔

GET /book/_search {"query": { "terms": { "tags": [ "search", "full_text", "dev" ] }} } ?

3.7、exist query

查詢有某些字段值的文檔

GET /_search {"query": {"exists": {"field": "join_date"}} }

3. 8、Fuzzy query

返回包含與搜索詞類似的詞的文檔，該詞由Levenshtein編輯距離度量。

包括以下幾種情況：

• 更改角色（box→fox）

• 刪除字符（aple→apple）

• 插入字符（sick→sic）

• 調(diào)換兩個(gè)相鄰字符（ACT→CAT）

GET /book/_search {"query": {"fuzzy": {"description": {"value": "jave"}}} }

3.9、ids

GET /book/_search {"query": {"ids" : {"values" : ["1", "4", "100"]}} }

3.10、prefix 前綴查詢

GET /book/_search {"query": {"prefix": {"description": {"value": "spring"}}} }

3.11、regexp query 正則查詢

GET /book/_search {"query": {"regexp": {"description": {"value": "j.*a","flags" : "ALL","max_determinized_states": 10000,"rewrite": "constant_score"}}} }

4． Filter

4.1 filter與query示例

需求：用戶查詢description中有"java程序員"，并且價(jià)格大于80小于90的數(shù)據(jù)。

GET /book/_search {"query": {"bool": {"must": [{"match": {"description": "java程序員"}},{"range": {"price": {"gte": 80,"lte": 90}}}]}} }

使用filter:

GET /book/_search {"query": {"bool": {"must": [{"match": {"description": "java程序員"}}],"filter": {"range": {"price": {"gte": 80,"lte": 90}}}}} }

4.2 filter與query對(duì)比

filter，僅僅只是按照搜索條件過濾出需要的數(shù)據(jù)而已，不計(jì)算任何相關(guān)度分?jǐn)?shù)，對(duì)相關(guān)度沒有任何影響。

query，會(huì)去計(jì)算每個(gè)document相對(duì)于搜索條件的相關(guān)度，并按照相關(guān)度進(jìn)行排序。

應(yīng)用場(chǎng)景：

一般來說，如果你是在進(jìn)行搜索，需要將最匹配搜索條件的數(shù)據(jù)先返回，那么用query 如果你只是要根據(jù)一些條件篩選出一部分?jǐn)?shù)據(jù)，不關(guān)注其排序，那么用filter

4.3 filter與query性能

filter，不需要計(jì)算相關(guān)度分?jǐn)?shù)，不需要按照相關(guān)度分?jǐn)?shù)進(jìn)行排序，同時(shí)還有內(nèi)置的自動(dòng)cache最常使用filter的數(shù)據(jù)

query，相反，要計(jì)算相關(guān)度分?jǐn)?shù)，按照分?jǐn)?shù)進(jìn)行排序，而且無法cache結(jié)果

5． 定位錯(cuò)誤語(yǔ)法

驗(yàn)證錯(cuò)誤語(yǔ)句：

GET /book/_validate/query?explain {"query": {"mach": {"description": "java程序員"}} }

返回：

{"valid" : false,"error" : "org.elasticsearch.common.ParsingException: no [query] registered for [mach]" }

正確

GET /book/_validate/query?explain {"query": {"match": {"description": "java程序員"}} }

返回

{"_shards" : {"total" : 1,"successful" : 1,"failed" : 0},"valid" : true,"explanations" : [{"index" : "book","valid" : true,"explanation" : "description:java description:程序員"}] }

一般用在那種特別復(fù)雜龐大的搜索下，比如你一下子寫了上百行的搜索，這個(gè)時(shí)候可以先用validate api去驗(yàn)證一下，搜索是否合法。

合法以后，explain就像mysql的執(zhí)行計(jì)劃，可以看到搜索的目標(biāo)等信息。

6． 定制排序規(guī)則

6.1 默認(rèn)排序規(guī)則

默認(rèn)情況下，是按照_score降序排序的

然而，某些情況下，可能沒有用到_score，比如說filter

GET book/_search {"query": {"bool": {"must": [{"match": {"description": "java程序員"}}]}} }

當(dāng)然，也可以是constant_score

6.2 定制排序規(guī)則

相當(dāng)于sql中order by ?sort=sprice:desc

GET /book/_search {"query": {"constant_score": {"filter" : {"term" : {"studymodel" : "201001"}}}},"sort": [{"price": {"order": "asc"}}] }

7． Text字段排序問題

如果對(duì)一個(gè)text field進(jìn)行排序，結(jié)果往往不準(zhǔn)確，因?yàn)榉衷~后是多個(gè)單詞，再排序就不是我們想要的結(jié)果了。

通常解決方案是，將一個(gè)text field建立兩次索引，一個(gè)分詞，用來進(jìn)行搜索；一個(gè)不分詞，用來進(jìn)行排序。

fielddate:true

PUT /website {"mappings": {"properties": {"title": {"type": "text","fields": {"keyword": {"type": "keyword"} ? ? ? ?} ? ? ?},"content": {"type": "text"},"post_date": {"type": "date"},"author_id": {"type": "long"}}} }

插入數(shù)據(jù)

PUT /website/_doc/1 {"title": "first article","content": "this is my second article","post_date": "2019-01-01","author_id": 110 } ? PUT /website/_doc/2 {"title": "second article","content": "this is my second article","post_date": "2019-01-01","author_id": 110 } ? PUT /website/_doc/3 {"title": "third article","content": "this is my third article","post_date": "2019-01-02","author_id": 110 }

搜索

GET /website/_search {"query": {"match_all": {}},"sort": [{"title.keyword": {"order": "desc"}}] }

8． Scroll分批查詢

場(chǎng)景：下載某一個(gè)索引中1億條數(shù)據(jù)，到文件或是數(shù)據(jù)庫(kù)。

不能一下全查出來，系統(tǒng)內(nèi)存溢出。所以使用scoll滾動(dòng)搜索技術(shù)，一批一批查詢。

scoll搜索會(huì)在第一次搜索的時(shí)候，保存一個(gè)當(dāng)時(shí)的視圖快照，之后只會(huì)基于該舊的視圖快照提供數(shù)據(jù)搜索，如果這個(gè)期間數(shù)據(jù)變更，是不會(huì)讓用戶看到的

每次發(fā)送scroll請(qǐng)求，我們還需要指定一個(gè)scoll參數(shù)，指定一個(gè)時(shí)間窗口，每次搜索請(qǐng)求只要在這個(gè)時(shí)間窗口內(nèi)能完成就可以了。

搜索

GET /book/_search?scroll=1m {"query": {"match_all": {}},"size": 3 }

返回

{"_scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAMOkWTURBNDUtcjZTVUdKMFp5cXloVElOQQ==","took" : 3,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 3,"relation" : "eq"},"max_score" : 1.0,"hits" : []} }

獲得的結(jié)果會(huì)有一個(gè)scoll_id，下一次再發(fā)送scoll請(qǐng)求的時(shí)候，必須帶上這個(gè)scoll_id

GET /_search/scroll {"scroll": "1m", "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAMOkWTURBNDUtcjZTVUdKMFp5cXloVElOQQ==" }

與分頁(yè)區(qū)別：

分頁(yè)給用戶看的 deep paging

scroll是用戶系統(tǒng)內(nèi)部操作，如下載批量數(shù)據(jù)，數(shù)據(jù)轉(zhuǎn)移。零停機(jī)改變索引映射。

總結(jié)

以上是生活随笔為你收集整理的Elasticsearch之Query DSL语法入门的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。