听某个老师的ElasticSearch记的笔记了
生活随笔
收集整理的這篇文章主要介紹了
听某个老师的ElasticSearch记的笔记了
小編覺(jué)得挺不錯(cuò)的,現(xiàn)在分享給大家,幫大家做個(gè)參考.
1.1ES簡(jiǎn)介
ES是使用java 語(yǔ)言并且基于lucence編寫(xiě)的搜索引擎框架，他提供了分布式的全文搜索功能，提供了一個(gè)統(tǒng)一的基于restful風(fēng)格的web 接口。
lucence:一個(gè)搜索引擎底層
分布式：突出ES的橫向擴(kuò)展能力
全文檢索：將一段詞語(yǔ)進(jìn)行分詞，并將分出的詞語(yǔ)統(tǒng)一的放在一個(gè)分詞庫(kù)中，再搜索時(shí)，根據(jù)關(guān)鍵字取分詞庫(kù)中檢索，找到匹配的內(nèi)容（倒排索引）。
restful風(fēng)格的web 接口：只要發(fā)送一個(gè)http請(qǐng)求，并且根據(jù)請(qǐng)求方式的不同，攜帶參數(shù)的不同，執(zhí)行相應(yīng)的功能。
應(yīng)用廣泛：WIKI, github,Gold man
1.2ES的由來(lái)
回憶時(shí)光**許多年前，一個(gè)剛結(jié)婚的名叫 Shay Banon 的失業(yè)開(kāi)發(fā)者，跟著他的妻子去了倫敦，他的妻子在那里學(xué)習(xí)廚師。 在尋找一個(gè)賺錢的工作的時(shí)候，為了給他的妻子做一個(gè)食譜搜索引擎，他開(kāi)始使用 Lucene 的一個(gè)早期版本。直接使用 Lucene 是很難的，因此 Shay 開(kāi)始做一個(gè)抽象層，Java 開(kāi)發(fā)者使用它可以很簡(jiǎn)單的給他們的程序添加搜索功能。 他發(fā)布了他的第一個(gè)開(kāi)源項(xiàng)目 Compass。后來(lái) Shay 獲得了一份工作，主要是高性能，分布式環(huán)境下的內(nèi)存數(shù)據(jù)網(wǎng)格。這個(gè)對(duì)于高性能，實(shí)時(shí)，分布式搜索引擎的需求尤為突出， 他決定重寫(xiě) Compass，把它變?yōu)橐粋€(gè)獨(dú)立的服務(wù)并取名 Elasticsearch。第一個(gè)公開(kāi)版本在2010年2月發(fā)布，從此以后，Elasticsearch 已經(jīng)成為了 Github 上最活躍的項(xiàng)目之一，他擁有超過(guò)300名 contributors(目前736名 contributors )。 一家公司已經(jīng)開(kāi)始圍繞 Elasticsearch 提供商業(yè)服務(wù)，并開(kāi)發(fā)新的特性，但是，Elasticsearch 將永遠(yuǎn)開(kāi)源并對(duì)所有人可用。據(jù)說(shuō)，Shay 的妻子還在等著她的食譜搜索引擎…1.3ES和solr
1.solr 查詢死數(shù)據(jù)，速度比es快。但是數(shù)據(jù)如果是改變的，solr查詢速度會(huì)降低很多，ES的查詢速度沒(méi)有明顯的改變
2.solr搭建集群 依賴ZK，ES本身就支持集群搭建
3.最開(kāi)始solr 的社區(qū)很火爆，針對(duì)國(guó)內(nèi)文檔 少，ES出現(xiàn)后，國(guó)內(nèi)社區(qū)火爆程度 上升，，ES的文檔非常健全
4.ES對(duì)云計(jì)算和大數(shù)據(jù)支持很好
1.4倒排索引
1.將存放的數(shù)據(jù)以一定的方式進(jìn)行分詞，并將分詞的內(nèi)容存放到一個(gè)單獨(dú)的分詞庫(kù)中。
2.當(dāng)用戶取查詢數(shù)據(jù)時(shí)，會(huì)將用戶的查詢關(guān)鍵字進(jìn)行分詞，然后去分詞庫(kù)中匹配內(nèi)容，最終得到數(shù)據(jù)的id標(biāo)識(shí)
3.根據(jù)id標(biāo)識(shí)去存放數(shù)據(jù)的位置拉去指定數(shù)據(jù)
2 安裝
2.1 elasticsearch 安裝
http://hub.daocloud.io/ docker 鏡像工廠地址
version: "3.1" services:elasticsearch: image: daocloud.io/library/elasticsearch:6.5.4restart: alwayscontainer_name: elasticsearchports: - 9200:9200- 9300:9300kibana:image: daocloud.io/library/kibana:6.5.4restart: alwayscontainer_name: kibanaports: - 9200:9200environment:- elasticsearch_url=ip:9200depends_on:- elasticseatch或者本地下載
2.2
https://github.com/medcl/elasticsearch-analysis-ik/archive/v6.8.10.zip
官方給的安裝的辦法 ./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.3.0/elasticsearch-analysis-ik-6.3.0.zip本地自己安裝
https://github.com/medcl/elasticsearch-analysis-ik/archive/v6.8.10.zip 下載好后， 執(zhí)行 mvn clean package 打包（注意pom文件中的es的版本，如果和自己的es的版本不一致，手動(dòng)改下） elasticsearch-analysis-ik-6.8.10\target\releases 中壓縮包的內(nèi)容copy到 elasticsearch-6.8.10\plugins\ik 下kibana 主要用到 Dev Tools 和 Management
POST _analyze {"analyzer": "ik_max_word","text":"我是中國(guó)人" }3 es的基本操作
3.1 es的結(jié)構(gòu)
3.1.1索引indx，分片，備份
ES服務(wù)中會(huì)創(chuàng)建多個(gè)索引 每個(gè)縮影默認(rèn)被分成5個(gè)分片 每個(gè)分片存在至少一個(gè)備份分片 備份分片 不會(huì)幫助檢索數(shù)據(jù)（當(dāng)ES檢索壓力特別大的時(shí)候才，備份分片才會(huì)幫助檢索數(shù)據(jù)） 備份的分片必須放在不同的服務(wù)器中3.1.2 類型type
一個(gè)索引下可以創(chuàng)建多個(gè)類型 PS:版本不同,類型的創(chuàng)建也不同3.1.3 文檔document
一個(gè)類型下可以有多個(gè)文檔，這個(gè)文檔就相當(dāng)于mysql表中的多行數(shù)據(jù)3.1.4 屬性field
一個(gè)文檔中可以包含多個(gè)屬性，類似于mysql 表中的一行數(shù)據(jù)有多個(gè)列3.2操作ES的restful語(yǔ)法
GET請(qǐng)求：http://ip:port/index :查詢索引信息http://ip:port/index/type/doc_id :查詢指定的文檔信息 POST請(qǐng)求：http://ip:port/index/type/_search: 查詢文檔，可以在請(qǐng)求體中添加json字符串來(lái)代表查詢條件http://ip:port/index/type/doc_id/_update: 修改文檔，在請(qǐng)求體中添加json字符串來(lái)代表修改的信息 PUT請(qǐng)求：http://ip:port/index : 創(chuàng)建一個(gè)索引，需要在請(qǐng)求體中指定索引的信息http://ip:port/index/type/_mappings:代表創(chuàng)建索引時(shí)，指定索引文檔存儲(chǔ)屬性的信息 DELETE 請(qǐng)求：http://ip:port/index： 刪除跑路http://ip:port/index/type/doc_id: 刪除指定的文檔3.3 索引的操作
3.3.1 創(chuàng)建一個(gè)索引
#創(chuàng)建一個(gè)索引 #number_of_shards 分片 #number_of_replicas 備份 PUT /person {"settings": {"number_of_shards": 5, "number_of_replicas": 1} }3.3.2 查看一個(gè)索引
1.management2. #查看索引信息 GET /person3.3.3 刪除 索引
1.management2. #刪除索引 DELETE /person3.4 ES中Field可以指定的類型
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/mapping-types.html 官方 文檔
字符串類型:text: 一般用于全文檢索，將當(dāng)前field 進(jìn)行分詞keyword:當(dāng)前field 不會(huì)進(jìn)行分詞 數(shù)值類型：long:Intger:short:byte:double:float:half_float: 精度比f(wàn)loat 小一半scaled_float:根據(jù)一個(gè)long 和scaled 來(lái)表達(dá)一個(gè)浮點(diǎn)型 long-345, -scaled 100 ->3.45 時(shí)間類型：date類型,根據(jù)時(shí)間類型指定具體的格式PUT my_index{"mappings": {"_doc": {"properties": {"date": {"type": "date","format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"}}}}} 布爾類型：boolean 類型，表達(dá)true 和false 二進(jìn)制類型：binary類型暫時(shí)支持Base64編碼的字符串 范圍類型：integer_range：float_range：long_range：賦值時(shí)，無(wú)需指定具體的內(nèi)容，只需存儲(chǔ)一個(gè)范圍即可，gte,lte,gt,lt,double_range：date_range：ip_range：PUT range_index{"settings": {"number_of_shards": 2},"mappings": {"_doc": {"properties": {"expected_attendees": {"type": "integer_range"},"time_frame": {"type": "date_range", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"}}}}}PUT range_index/_doc/1?refresh{"expected_attendees" : { "gte" : 10,"lte" : 20},"time_frame" : { "gte" : "2015-10-31 12:00:00", "lte" : "2015-11-01"}} 經(jīng)緯度類型：geo_point:用來(lái)存儲(chǔ)經(jīng)緯度 IP類型：ip:可以存儲(chǔ)IPV4 和IPV6 其他的數(shù)據(jù)類型，參考官網(wǎng)3.5 創(chuàng)建索引并指定數(shù)據(jù)結(jié)構(gòu)
#創(chuàng)建索引，指定數(shù)據(jù)類型 PUT /book {"settings": {#分片數(shù)"number_of_shards": 5,#備份數(shù)"number_of_replicas": 1},#指定數(shù)據(jù)類型"mappings": {#類型 Type"novel":{#文檔存儲(chǔ)的field"properties":{#field屬性名"name":{#類型"type":"text",#指定分詞器"analyzer":"ik_max_word",#指定當(dāng)前的field可以被作為查詢的條件"index":true,#是否需要額外存儲(chǔ)"store":false},"author":{"type":"keyword"},"count":{"type":"long"},"on-sale":{"type":"date",#指定時(shí)間類型的格式化方式"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"},"descr":{"type":"text","analyzer":"ik_max_word"}}}} }3.6 文檔操作
文檔在ES服務(wù)中的唯一標(biāo)識(shí)， _indx ,_type,_id 三個(gè)內(nèi)容為組合，鎖定一個(gè)文檔，操作時(shí)添加還時(shí)修改操作，3.6.1 新建文檔
自動(dòng)生成id #添加文檔，自動(dòng)生成id POST /book/novel {"name":"盤(pán)龍","author":"我吃西紅柿","count":100000,"on-sale":"2001-01-01","descr":"大小的血睛鬃毛獅，力大無(wú)窮的紫睛金毛猿，毀天滅地的九頭蛇皇，攜帶著毀滅雷電的恐怖雷龍……這里無(wú)奇不有，這是一個(gè)廣博的魔幻世界。強(qiáng)者可以站在黑色巨龍的頭頂遨游天際，恐怖的魔法可以焚燒江河，可以毀滅城池，可以夷平山岳……" }#添加文檔,手動(dòng)指定id PUT /book/novel/1 {"name":"紅樓夢(mèng)","author":"曹雪芹","count":10000000,"on-sale":"2501-01-01","descr":"中國(guó)古代章回體長(zhǎng)篇小說(shuō)，中國(guó)古典四大名著之一，一般認(rèn)為是清代作家曹雪芹所著。小說(shuō)以賈、史、王、薛四大家族的興衰為背景，以富貴公子賈寶玉為視角，以賈寶玉與林黛玉、薛寶釵的愛(ài)情婚姻悲劇為主線，描繪了一批舉止見(jiàn)識(shí)出于須眉之上的閨閣佳人的人生百態(tài)，展現(xiàn)了真正的人性美和悲劇美" }3.6.2 修改文檔
1.覆蓋式修改
#添加文檔,手動(dòng)指定id PUT /book/novel/1 {"name":"紅樓夢(mèng)","author":"曹雪芹","count":1000444,"on-sale":"2501-01-01","descr":"中國(guó)古代章回體長(zhǎng)篇小說(shuō)，中國(guó)古典四大名著之一，一般認(rèn)為是清代作家曹雪芹所著。小說(shuō)以賈、史、王、薛四大家族的興衰為背景，以富貴公子賈寶玉為視角，以賈寶玉與林黛玉、薛寶釵的愛(ài)情婚姻悲劇為主線，描繪了一批舉止見(jiàn)識(shí)出于須眉之上的閨閣佳人的人生百態(tài)，展現(xiàn)了真正的人性美和悲劇美" }2.使用doc修改方式
#修改文檔，使用doc 方式 POST /book/novel/1/_update {"doc":{#指定需要修改的field和對(duì)應(yīng)的值"count":566666} }3.6.3 刪除文檔
#根據(jù)id刪除文檔 DELETE /book/novel/3mEnk3MBaSKoGN4T2olw4.java 操作ElasticSearch
4.1 java 連接ES
創(chuàng)建maven工程 導(dǎo)入依賴 <!-- 1.elasticsearch--><dependency><groupId>org.elasticsearch</groupId><artifactId>elasticsearch</artifactId><version>6.8.10</version></dependency> <!-- 2.elasticsearch 高級(jí)API--><dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId><version>6.8.10</version></dependency> <!-- 3.junit--><dependency><groupId>junit</groupId><artifactId>junit</artifactId><version>4.12</version></dependency> <!-- 4.lombok--><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.16.22</version></dependency>創(chuàng)建client鏈接
package com.utils;import org.apache.http.HttpHost; import org.elasticsearch.client.RestClient; import org.elasticsearch.client.RestClientBuilder; import org.elasticsearch.client.RestHighLevelClient;public class EsClient {public static RestHighLevelClient getClient(){// 創(chuàng)建 HttpHostHttpHost httpHost = new HttpHost("127.0.0.1",9200);// 創(chuàng)建 RestClientBuilderRestClientBuilder builder = RestClient.builder(httpHost);// 創(chuàng)建 RestHighLevelClientRestHighLevelClient client = new RestHighLevelClient(builder);return client;} }4.2創(chuàng)建索引
package com.test;import com.utils.EsClient; import org.elasticsearch.action.admin.indices.create.CreateIndexRequest; import org.elasticsearch.action.admin.indices.create.CreateIndexResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.json.JsonXContent; import org.junit.Test;public class Demo2 {RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void createIndx() throws Exception{// 1.準(zhǔn)備關(guān)于索引的settingSettings.Builder settings = Settings.builder().put("number_of_shards", 2).put("number_of_replicas", 1);// 2.準(zhǔn)備關(guān)于索引的mappingXContentBuilder mappings = JsonXContent.contentBuilder().startObject().startObject("properties").startObject("name").field("type", "text").endObject().startObject("age").field("type", "integer").endObject().startObject("birthday").field("type", "date").field("format", "yyyy-MM-dd").endObject().endObject().endObject();// 3.將settings和mappings 封裝到到一個(gè)Request對(duì)象中CreateIndexRequest request = new CreateIndexRequest(index).settings(settings).mapping(type,mappings);// 4.使用client 去連接ESCreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);System.out.println("response:"+response.toString());} }4.3 檢查索引是否存在，刪除索引
4.3.1 檢查索引存在
package com.test;import com.utils.EsClient; import org.elasticsearch.action.admin.indices.create.CreateIndexRequest; import org.elasticsearch.action.admin.indices.create.CreateIndexResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.client.indices.GetIndexRequest; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.json.JsonXContent; import org.junit.Test;import java.io.IOException;public class Demo2 {RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void existTest() throws IOException {// 1.準(zhǔn)備request 對(duì)象GetIndexRequest request = new GetIndexRequest(index);// 2.通過(guò)client 去 操作boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);// 3輸出結(jié)果System.out.println(exists);} }4.3.2 刪除索引
package com.test;import com.utils.EsClient; import org.elasticsearch.action.admin.indices.create.CreateIndexRequest; import org.elasticsearch.action.admin.indices.create.CreateIndexResponse; import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest; import org.elasticsearch.action.support.master.AcknowledgedResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.client.indices.GetIndexRequest; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.json.JsonXContent; import org.junit.Test;import java.io.IOException;public class Demo2 {RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void testDelete() throws IOException {// 1.獲取requestDeleteIndexRequest request = new DeleteIndexRequest(index);// 2.使用client 操作requestAcknowledgedResponse delete = client.indices().delete(request, RequestOptions.DEFAULT);// 3.輸出結(jié)果System.out.println(delete.isAcknowledged());} }4.4 Java操作文檔
4.4.1 添加文檔操作
public class Demo3 {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void createDocTest() throws IOException {// 1.準(zhǔn)備一個(gè)json數(shù)據(jù)Person person = new Person(1,"張三",33,new Date());String json = mapper.writeValueAsString(person);// 2.創(chuàng)建一個(gè)request對(duì)象(手動(dòng)指定的方式創(chuàng)建)IndexRequest request = new IndexRequest(index,type,person.getId().toString());request.source(json, XContentType.JSON);// 3.使用client 操作request對(duì)象生成docIndexResponse response = client.index(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果System.out.println(response.getResult().toString());} }4.4.2 修改文檔
public class Demo3 {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void updateDocTest() throws Exception{// 1.創(chuàng)建要跟新的MapMap<String,Object> doc = new HashMap<>();doc.put("name","張三三");// 2.創(chuàng)建request, 將doc 封裝進(jìn)去UpdateRequest request = new UpdateRequest(index,type,"1");request.doc(doc);// 3. client 去操作 requestUpdateResponse response = client.update(request, RequestOptions.DEFAULT);// 4.輸出 更新結(jié)果System.out.println(response.getResult());} }4.4.3 刪除文檔
public class Demo3 {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void deleteDocTest() throws Exception{// 1.封裝刪除對(duì)象DeleteRequest request = new DeleteRequest(index,type,"1");// 2 client 操作 request對(duì)象DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);// 3.輸出結(jié)果System.out.println(response.getResult().toString());} }4.5 java批量操作文檔
批量操作 新增
@Testpublic void bulkCreateDoc() throws Exception{// 1.準(zhǔn)備多個(gè)json 對(duì)象Person p1 = new Person(1,"張三",23,new Date());Person p2 = new Person(2,"里斯",24,new Date());Person p3 = new Person(3,"王武",24,new Date());String json1 = mapper.writeValueAsString(p1);String json2 = mapper.writeValueAsString(p2);String json3 = mapper.writeValueAsString(p3);// 2.創(chuàng)建requestBulkRequest bulkRequest = new BulkRequest();bulkRequest.add(new IndexRequest(index,type,p1.getId().toString()).source(json1,XContentType.JSON)).add(new IndexRequest(index,type,p2.getId().toString()).source(json2,XContentType.JSON)).add(new IndexRequest(index,type,p3.getId().toString()).source(json3,XContentType.JSON));// 3.client 執(zhí)行BulkResponse responses = client.bulk(bulkRequest, RequestOptions.DEFAULT);// 4.輸出結(jié)果System.out.println(responses.getItems().toString());}批量刪除
public void bulkDelete() throws Exception{// 1.創(chuàng)建Request 對(duì)象BulkRequest bulkRequest = new BulkRequest();bulkRequest.add(new DeleteRequest(index,type,"1"));bulkRequest.add(new DeleteRequest(index,type,"2"));bulkRequest.add(new DeleteRequest(index,type,"3"));// 2.執(zhí)行BulkResponse re = client.bulk(bulkRequest, RequestOptions.DEFAULT);// 3.輸出結(jié)果System.out.println(re.toString());}5.ES 練習(xí)
索引：sms-logs-index 類型：sms-logs-type public class Demo4 {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void createIndex() throws Exception{// 1.準(zhǔn)備關(guān)于索引的settingSettings.Builder settings = Settings.builder().put("number_of_shards", 3).put("number_of_replicas", 1);// 2.準(zhǔn)備關(guān)于索引的mappingXContentBuilder mappings = JsonXContent.contentBuilder().startObject().startObject("properties").startObject("corpName").field("type", "keyword").endObject().startObject("createDate").field("type", "date").field("format", "yyyy-MM-dd").endObject().startObject("fee").field("type", "long").endObject().startObject("ipAddr").field("type", "ip").endObject().startObject("longCode").field("type", "keyword").endObject().startObject("mobile").field("type", "keyword").endObject().startObject("operatorId").field("type", "integer").endObject().startObject("province").field("type", "keyword").endObject().startObject("replyTotal").field("type", "integer").endObject().startObject("sendDate").field("type", "date").field("format", "yyyy-MM-dd").endObject().startObject("smsContent").field("type", "text").field("analyzer", "ik_max_word").endObject().startObject("state").field("type", "integer").endObject().endObject().endObject();// 3.將settings和mappings 封裝到到一個(gè)Request對(duì)象中CreateIndexRequest request = new CreateIndexRequest(index).settings(settings).mapping(type,mappings);// 4.使用client 去連接ESCreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);System.out.println("response:"+response.toString());}@Testpublic void bulkCreateDoc() throws Exception{// 1.準(zhǔn)備多個(gè)json 對(duì)象String longcode = "1008687";String mobile ="138340658";List<String> companies = new ArrayList<>();companies.add("騰訊課堂");companies.add("阿里旺旺");companies.add("海爾電器");companies.add("海爾智家公司");companies.add("格力汽車");companies.add("蘇寧易購(gòu)");List<String> provinces = new ArrayList<>();provinces.add("北京");provinces.add("重慶");provinces.add("上海");provinces.add("晉城");BulkRequest bulkRequest = new BulkRequest();for (int i = 1; i <16 ; i++) {Thread.sleep(1000);SmsLogs s1 = new SmsLogs();s1.setId(i);s1.setCreateDate(new Date());s1.setSendDate(new Date());s1.setLongCode(longcode+i);s1.setMobile(mobile+2*i);s1.setCorpName(companies.get(i%5));s1.setSmsContent(SmsLogs.doc.substring((i-1)*100,i*100));s1.setState(i%2);s1.setOperatorId(i%3);s1.setProvince(provinces.get(i%4));s1.setIpAddr("127.0.0."+i);s1.setReplyTotal(i*3);s1.setFee(i*6+"");String json1 = mapper.writeValueAsString(s1);bulkRequest.add(new IndexRequest(index,type,s1.getId().toString()).source(json1, XContentType.JSON));System.out.println("數(shù)據(jù)"+i+s1.toString());}// 3.client 執(zhí)行BulkResponse responses = client.bulk(bulkRequest, RequestOptions.DEFAULT);// 4.輸出結(jié)果System.out.println(responses.getItems().toString());} }6.ElasticSearch的各種查詢
6.1 term 和terms 查詢
6.1.1 term 查詢
term 查詢是代表完全匹配，搜索之前不會(huì)對(duì)你搜索的關(guān)鍵字進(jìn)行分詞，直接拿 關(guān)鍵字 去文檔分詞庫(kù)中匹配內(nèi)容 #term查詢 POST /sms-logs-index/sms-logs-type/_search {#limit ?"from": 0, #limit x,?"size":5,"query": {"term": {"province": {"value": "北京"}}} } public class TermSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void termSearchTest() throws IOException {// 1.創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();builder.from(0);builder.size(5);builder.query(QueryBuilders.termQuery("province","北京"));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {Map<String, Object> sourceAsMap = hit.getSourceAsMap();System.out.println(sourceAsMap);}} }6.1.2 查詢
terms 和 term 查詢的機(jī)制一樣，搜索之前不會(huì)對(duì)你搜索的關(guān)鍵字進(jìn)行分詞，直接拿 關(guān)鍵字 去文檔分詞庫(kù)中匹配內(nèi)容 terms:是針對(duì)一個(gè)字段包含多個(gè)值 term : where province =北京 terms: where province = 北京 or province =? (類似于mysql 中的 in) 也可針對(duì) text, 只是在分詞庫(kù)中查詢的時(shí)候不會(huì)進(jìn)行分詞 #terms 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"terms": {"province": ["北京","晉城"]}} } public class TermSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void termsSearchTest() throws IOException {// 1.創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();builder.query(QueryBuilders.termsQuery("province","北京","晉城"));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}} }6.2 match
match 查詢屬于高級(jí)查詢，會(huì)根據(jù)你查詢字段的類型不一樣，采用不同的查詢方式 查詢的是日期或者數(shù)值，他會(huì)將你基于字符串的查詢內(nèi)容轉(zhuǎn)換為日期或數(shù)值對(duì)待如果查詢的內(nèi)容是一個(gè)不能被分詞的內(nèi)容（keyword）,match 不會(huì)將你指定的關(guān)鍵字進(jìn)行分詞如果查詢的內(nèi)容是一個(gè)可以被分詞的內(nèi)容（text）,match 查詢會(huì)將你指定的內(nèi)容根據(jù)一定的方式進(jìn)行分詞，去分詞庫(kù)中匹配指定的內(nèi)容 match 查詢，實(shí)際底層就是多個(gè)term 查詢，將多個(gè)term查詢的結(jié)果給你封裝到一起6.2.1 math_all
查詢?nèi)績(jī)?nèi)容，不指定查詢條件 #match_all 查詢 POST /sms-logs-index/sms-logs-type/_search {"query":{"match_all": {}} } public class MatchSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void matchAllSearch() throws IOException {// 1.創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();builder.query(QueryBuilders.matchAllQuery());// ES 默認(rèn)只查詢10條數(shù)據(jù)builder.size(20);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}System.out.println(response.getHits().getHits().length);} }6.2.2 match 查詢
指定一個(gè)field 作為查詢條件
#match 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"smsContent": "偉大戰(zhàn)士"}} } public class MatchSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void matchSearch() throws IOException {// 1.創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------------------builder.query(QueryBuilders.matchQuery("smsContent","偉大戰(zhàn)士"));//--------------------------------------------------------------builder.size(20);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}System.out.println(response.getHits().getHits().length);}}6.2.3 布爾match 查詢
基于一個(gè)field 匹配的內(nèi)容，按照 and 或者or的方式連接 #布爾match查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"smsContent": {# 既包含 戰(zhàn)士 也包含 團(tuán)隊(duì)"query": "戰(zhàn)士 團(tuán)隊(duì)","operator": "and"}}} }#布爾match查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"smsContent": {# 既包含 戰(zhàn)士 或者 團(tuán)隊(duì)"query": "戰(zhàn)士 團(tuán)隊(duì)","operator": "or"}}} } @Testpublic void booleanMatchSearch() throws IOException {// 1.創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------------------builder.query(QueryBuilders.matchQuery("smsContent","戰(zhàn)士 團(tuán)隊(duì)").operator(Operator.AND));//--------------------------------------------------------------builder.size(20);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}System.out.println(response.getHits().getHits().length);}6.2.4 multi_match
match 針對(duì)一個(gè)field 做檢索，multi_math 針對(duì)多個(gè)field 進(jìn)行檢索，多個(gè)field對(duì)應(yīng)一個(gè)文本。 #multi_math 查詢 POST /sms-logs-index/sms-logs-type/_search {"query":{"multi_match": {"query": "北京","fields": ["province","smsContent"]}} } public void multiMatchSearch() throws IOException {// 1.創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------------------builder.query(QueryBuilders.multiMatchQuery("北京","province","smsContent"));//--------------------------------------------------------------builder.size(20);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}System.out.println(response.getHits().getHits().length);}6.3 其他查詢
6.3.1id 查詢
#id 查詢 GET /sms-logs-index/sms-logs-type/1 public class IdGetSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void findById() throws IOException {// 創(chuàng)建GetRequest對(duì)象GetRequest request = new GetRequest(index,type,"1");// 執(zhí)行查詢GetResponse response = client.get(request, RequestOptions.DEFAULT);// 輸出結(jié)果System.out.println(response.getSourceAsMap());} }6.3.2 ids查詢
根據(jù)多個(gè)id 查詢,類似 mysql 中的 where in (id1,id2...) #ids 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"ids": {"values": ["1","2","3"]}} } public void findByIds() throws IOException {// 創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.idsQuery().addIds("1","2","3"));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}6.3.3 prefix 查詢
前綴查詢，可以通過(guò)一個(gè)關(guān)鍵字去指定一個(gè)field 的前綴，從而查詢到指定文檔
#prefix 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"prefix": {"corpName": {"value": "海"}}} } #match 查詢 在這里是什么都查不到的 和上邊的prefix 做比較 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"corpName": "海"}} } public void findByPrefix() throws IOException {// 創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.prefixQuery("corpName","阿"));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}6.3.4 fuzzy 查詢
模糊查詢，我們可以輸入一個(gè)字符的大概，ES 可以根據(jù)輸入的大概去匹配內(nèi)容。查詢結(jié)果不穩(wěn)定 #fuzzy 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"fuzzy": {"corpName": {"value": "騰訊客堂",#指定前邊幾個(gè)字符是不允許出現(xiàn)錯(cuò)誤的"prefix_length": 2}}} } public void findByFuzzy() throws IOException {// 創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.fuzzyQuery("corpName","騰訊客堂").prefixLength(2));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}6.3.5 wildcard 查詢
通配查詢，同mysql中的like 是一樣的，可以在查詢時(shí)，在字符串中指定通配符*和占位符？ #wildcard 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"wildcard": {"corpName": {"value": "海爾*"}}} }#wildcard 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"wildcard": {"corpName": {"value": "海爾??"}}} } public void findByWildCard() throws IOException {// 創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.wildcardQuery("corpName","海爾*"));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}6.3.6 rang 查詢
范圍查詢，只針對(duì)數(shù)值類型，對(duì)一個(gè)field 進(jìn)行大于或者小于的范圍指定
#rang 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"range": {"fee": {"gte": 10,"lte": 20}}} } public void findByRang() throws IOException {// 創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.rangeQuery("fee").gt(10).lte(30));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}6.3.7 regexp 查詢
正則查詢，通過(guò)你編寫(xiě)的正則表達(dá)式去匹配內(nèi)容 Ps:prefix wildcard fuzzy 和regexp 查詢效率比較低 ,在要求效率比較高時(shí)，避免使用 #regexp 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"regexp": {"mobile": "138[0-9]{8}"}} } public void findByRegexp() throws IOException {// 創(chuàng)建request對(duì)象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.regexpQuery("mobile","138[0-9]{8}"));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());} }6.4 深分頁(yè) scrol l
ES 對(duì)from +size時(shí)又限制的，from +size 之和 不能大于1W,超過(guò)后 效率會(huì)十分低下 原理：from+size ES查詢數(shù)據(jù)的方式，第一步將用戶指定的關(guān)鍵詞進(jìn)行分詞，第二部將詞匯去分詞庫(kù)中進(jìn)行檢索，得到多個(gè)文檔id,第三步去各個(gè)分片中拉去數(shù)據(jù)， 耗時(shí)相對(duì)較長(zhǎng)第四步根據(jù)score 將數(shù)據(jù)進(jìn)行排序， 耗時(shí)相對(duì)較長(zhǎng)第五步根據(jù)from 和size 的值 將部分?jǐn)?shù)據(jù)舍棄，第六步，返回結(jié)果。scroll +size ES 查詢數(shù)據(jù)的方式第一步將用戶指定的關(guān)鍵詞進(jìn)行分詞，第二部將詞匯去分詞庫(kù)中進(jìn)行檢索，得到多個(gè)文檔id,第三步，將文檔的id放在一個(gè)上下文中第四步，根據(jù)指定的size去ES中檢索指定個(gè)數(shù)數(shù)據(jù)，拿完數(shù)據(jù)的文檔id,會(huì)從上下文中移除第五步，如果需要下一頁(yè)的數(shù)據(jù)，直接去ES的上下文中找后續(xù)內(nèi)容。第六步，循環(huán)第四步和第五步scroll 不適合做實(shí)時(shí)查詢。 #scroll 查詢,返回第一頁(yè)數(shù)據(jù)，并將文檔id信息存放在ES上下文中，并指定生存時(shí)間 POST /sms-logs-index/sms-logs-type/_search?scroll=1m {"query": {"match_all": {}},"size": 2,"sort": [{"fee": {"order": "desc"}}] }#根據(jù)scroll 查詢下一頁(yè)數(shù)據(jù) POST _search/scroll {"scroll_id":"DnF1ZXJ5VGhlbkZldGNoAwAAAAAAABbqFk04VlZ1cjlUU2t1eHpsQWNRY1YwWWcAAAAAAAAW7BZNOFZWdXI5VFNrdXh6bEFjUWNWMFlnAAAAAAAAFusWTThWVnVyOVRTa3V4emxBY1FjVjBZZw==","scroll":"1m" }#刪除scroll上下文中的數(shù)據(jù) DELETE _search/scroll/DnF1ZXJ5VGhlbkZldGNoAwAAAAAAABchFk04VlZ1cjlUU2t1eHpsQWNRY1YwWWcAAAAAAAAXIBZNOFZWdXI5VFNrdXh6bEFjUWNWMFlnAAAAAAAAFx8WTThWVnVyOVRTa3V4emxBY1FjVjBZZw== public class ScrollSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void scrollSearch() throws IOException {// 1.創(chuàng)建requestSearchRequest searchRequest = new SearchRequest(index);searchRequest.types(type);// 2.指定scroll信息,過(guò)期時(shí)間searchRequest.scroll(TimeValue.timeValueMinutes(1L));// 3.指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();builder.size(4);builder.sort("fee", SortOrder.DESC);searchRequest.source(builder);// 4.獲取返回結(jié)果scrollId,獲取sourceSearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);String scrollId = response.getScrollId();System.out.println("-------------首頁(yè)數(shù)據(jù)---------------------");for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}while (true){// 5.創(chuàng)建scroll requestSearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);// 6.指定scroll 有效時(shí)間scrollRequest.scroll(TimeValue.timeValueMinutes(1L));// 7.執(zhí)行查詢，返回查詢結(jié)果SearchResponse scroll = client.scroll(scrollRequest, RequestOptions.DEFAULT);// 8.判斷是否查詢到數(shù)據(jù)，查詢到輸出SearchHit[] searchHits = scroll.getHits().getHits();if(searchHits!=null && searchHits.length >0){System.out.println("-------------下一頁(yè)數(shù)據(jù)---------------------");for (SearchHit hit : searchHits) {System.out.println(hit.getSourceAsMap());}}else{// 9.沒(méi)有數(shù)據(jù)，結(jié)束System.out.println("-------------結(jié)束---------------------");break;}}// 10.創(chuàng)建 clearScrollRequestClearScrollRequest clearScrollRequest = new ClearScrollRequest();// 11.指定scrollIdclearScrollRequest.addScrollId(scrollId);//12.刪除scrollClearScrollResponse clearScrollResponse = client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);// 13.輸出結(jié)果System.out.println("刪除scroll:"+clearScrollResponse.isSucceeded());} }6.5 delete-by-query
根據(jù)term,match 等查詢方式去刪除大量索引 PS:如果你要?jiǎng)h除的內(nèi)容，時(shí)index下的大部分?jǐn)?shù)據(jù)，推薦創(chuàng)建一個(gè)新的index,然后把保留的文檔內(nèi)容，添加到全新的索引 #Delet-by-query 刪除 POST /sms-logs-index/sms-logs-type/_delete_by_query {"query": {"range": {"fee": {"lt": 20}}} } public void deleteByQuery() throws IOException {// 1.創(chuàng)建DeleteByQueryRequestDeleteByQueryRequest request = new DeleteByQueryRequest(index);request.types(type);// 2.指定條件request.setQuery(QueryBuilders.rangeQuery("fee").lt(20));// 3.執(zhí)行BulkByScrollResponse response = client.deleteByQuery(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果System.out.println(response.toString());}6.6 復(fù)合查詢
復(fù)合過(guò)濾器，將你的多個(gè)查詢條件 以一定的邏輯組合在一起，must:所有條件組合在一起，表示 and 的意思 must_not: 將must_not中的條件，全部都不匹配，表示not的意思 should:所有條件用should 組合在一起，表示or 的意思 #省是 晉城 或者北京 # 運(yùn)營(yíng)商不能是聯(lián)通 #smsContent 包含 戰(zhàn)士 和的 POST /sms-logs-index/sms-logs-type/_search {"query": {"bool": {"should": [{"term": {"province": {"value": "晉城"}}},{"term": {"province": {"value": "北京"}}}],"must_not": [{"term": {"operatorId": {"value": "2"}}}],"must": [{"match": {"smsContent": "戰(zhàn)士"}},{"match": {"smsContent": "的"}}]}} } public void boolSearch() throws IOException {// 1.創(chuàng)建 searchRequestSearchRequest request = new SearchRequest(index);request.types(type);// 2.指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();// #省是 晉城 或者北京boolQueryBuilder.should(QueryBuilders.termQuery("province","北京"));boolQueryBuilder.should(QueryBuilders.termQuery("province","晉城"));//# 運(yùn)營(yíng)商不能是聯(lián)通boolQueryBuilder.mustNot(QueryBuilders.termQuery("operatorId",2));//#smsContent 包含 戰(zhàn)士 和的boolQueryBuilder.must(QueryBuilders.matchQuery("smsContent","戰(zhàn)士"));boolQueryBuilder.must(QueryBuilders.matchQuery("smsContent","的"));builder.query(boolQueryBuilder);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}6.6.2 boosting 查詢
boosting 查詢可以幫助我們?nèi)ビ绊懖樵兒蟮膕corepositive:只有匹配上positive 查詢的內(nèi)容，才會(huì)被放到返回的結(jié)果集中negative: 如果匹配上了positive 也匹配上了negative, 就可以 降低這樣的文檔score.negative_boost:指定系數(shù),必須小于1 0.5 關(guān)于查詢時(shí)，分?jǐn)?shù)時(shí)如何計(jì)算的：搜索的關(guān)鍵字再文檔中出現(xiàn)的頻次越高，分?jǐn)?shù)越高指定的文檔內(nèi)容越短，分?jǐn)?shù)越高。我們?cè)偎阉鲿r(shí)，指定的關(guān)鍵字也會(huì)被分詞，這個(gè)被分詞的內(nèi)容，被分詞庫(kù)匹配的個(gè)數(shù)越多，分?jǐn)?shù)就越高。 #boosting 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"boosting": {"positive": {"match": {"smsContent": "戰(zhàn)士"}}, "negative": {"match": {"smsContent": "團(tuán)隊(duì)"}},"negative_boost": 0.2}} } public void boostSearch() throws IOException {// 1.創(chuàng)建 searchRequestSearchRequest request = new SearchRequest(index);request.types(type);// 2.指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();BoostingQueryBuilder boost = QueryBuilders.boostingQuery(QueryBuilders.matchQuery("smsContent", "戰(zhàn)士"),QueryBuilders.matchQuery("smsContent", "團(tuán)隊(duì)")).negativeBoost(0.2f);builder.query(boost);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}6.7 filter 查詢
query 查詢：根據(jù)你的查詢條件，去計(jì)算文檔的匹配度得到一個(gè)分?jǐn)?shù)，并根據(jù)分?jǐn)?shù)排序，不會(huì)做緩存的。
filter 查詢：根據(jù)查詢條件去查詢文檔，不去計(jì)算分?jǐn)?shù)，而且filter會(huì)對(duì)經(jīng)常被過(guò)濾的數(shù)據(jù)進(jìn)行緩存。
#filter 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"bool": {"filter": [{"term": {"corpName": "海爾智家公司"}},{"range":{"fee":{"lte":50}}}]}} } public void filter() throws IOException {// 1.searchRequestSearchRequest searchRequest = new SearchRequest(index);searchRequest.types(type);// 2.指定查詢條件SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();BoolQueryBuilder boolBuilder = QueryBuilders.boolQuery();boolBuilder.filter(QueryBuilders.termQuery("corpName","海爾智家公司"));boolBuilder.filter(QueryBuilders.rangeQuery("fee").gt(20));sourceBuilder.query(boolBuilder);searchRequest.source(sourceBuilder);// 3.執(zhí)行SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);// 4. 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());System.out.println(hit.getId()+"的分?jǐn)?shù)是："+hit.getScore());}}6.8 高亮查詢
高亮查詢就是用戶輸入的關(guān)鍵字，以一定特殊樣式展示給用戶，讓用戶知道為什么這個(gè)結(jié)果被檢索出來(lái) 高亮展示的數(shù)據(jù)，本身就是文檔中的一個(gè)field,單獨(dú)將field以highlight的形式返回給用戶 ES提供了一個(gè)highlight 屬性，他和query 同級(jí)別。frament_size: 指定高亮數(shù)據(jù)展示多少個(gè)字符回來(lái)pre_tags:指定前綴標(biāo)簽<front color="red">post_tags:指定后綴標(biāo)簽 </font> #highlight 高亮查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"smsContent": "團(tuán)隊(duì)"}},"highlight": {"fields": {"smsContent":{}},"pre_tags":"<font color='red'>","post_tags":"</font>","fragment_size":10} } public void highLightQuery() throws IOException {// 1.創(chuàng)建requestSearchRequest request = new SearchRequest(index);request.types(type);// 2.指定查詢條件，指定高亮SearchSourceBuilder builder = new SearchSourceBuilder();builder.query(QueryBuilders.matchQuery("smsContent","團(tuán)隊(duì)"));HighlightBuilder highlightBuilder = new HighlightBuilder();highlightBuilder.field("smsContent",10).preTags("<font colr='red'>").postTags("</font>");builder.highlighter(highlightBuilder);request.source(builder);// 3.執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);//4. 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getHighlightFields().get("smsContent"));}}6.9 聚合查詢
ES的聚合查詢和mysql 的聚合查詢類似，ES的聚合查詢相比mysql 要強(qiáng)大得多。ES提供的統(tǒng)計(jì)數(shù)據(jù)的方式多種多樣。 #ES 聚合查詢的RSTFul 語(yǔ)法 POST /index/type/_search {"aggs":{"(名字)agg":{"agg_type":{"屬性"："值"}}} }6.9.1 去重計(jì)數(shù)聚合查詢
去重計(jì)數(shù)，cardinality 先將返回的文檔中的一個(gè)指定的field進(jìn)行去重，統(tǒng)計(jì)一共有多少條 # 去重計(jì)數(shù) 查詢 province POST /sms-logs-index/sms-logs-type/_search {"aggs": {"provinceAgg": {"cardinality": {"field": "province"}}} } public void aggCardinalityC() throws IOException {// 1.創(chuàng)建requestSearchRequest request = new SearchRequest(index);request.types(type);// 2. 指定使用聚合查詢方式SearchSourceBuilder builder = new SearchSourceBuilder();builder.aggregation(AggregationBuilders.cardinality("provinceAgg").field("province"));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果Cardinality agg = response.getAggregations().get("provinceAgg");System.out.println(agg.getValue());}6.9.2 范圍統(tǒng)計(jì)
統(tǒng)計(jì)一定范圍內(nèi)出現(xiàn)的文檔個(gè)數(shù)，比如，針對(duì)某一個(gè)field 的值再0~100,100~200,200~300 之間文檔出現(xiàn)的個(gè)數(shù)分別是多少 范圍統(tǒng)計(jì) 可以針對(duì) 普通的數(shù)值，針對(duì)時(shí)間類型，針對(duì)ip類型都可以響應(yīng)。 數(shù)值 rang 時(shí)間 date_rang ip ip_rang #針對(duì)數(shù)值方式的范圍統(tǒng)計(jì) from 帶等于效果 ，to 不帶等于效果 POST /sms-logs-index/sms-logs-type/_search {"aggs": {"agg": {"range": {"field": "fee","ranges": [{"to": 30},{"from": 30,"to": 60},{"from": 60}]}}} } #時(shí)間方式統(tǒng)計(jì) POST /sms-logs-index/sms-logs-type/_search {"aggs": {"agg": {"date_range": {"field": "sendDate","format": "yyyy", "ranges": [{"to": "2000"},{"from": "2000"}]}}} } #ip 方式 范圍統(tǒng)計(jì) POST /sms-logs-index/sms-logs-type/_search {"aggs": {"agg": {"ip_range": {"field": "ipAddr","ranges": [{"to": "127.0.0.8"},{"from": "127.0.0.8"}]}}} } public void aggRang() throws IOException {// 1.創(chuàng)建requestSearchRequest request = new SearchRequest(index);request.types(type);// 2. 指定使用聚合查詢方式SearchSourceBuilder builder = new SearchSourceBuilder();builder.aggregation(AggregationBuilders.range("agg").field("fee").addUnboundedTo(30).addRange(30,60).addUnboundedFrom(60));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果Range agg = response.getAggregations().get("agg");for (Range.Bucket bucket : agg.getBuckets()) {String key = bucket.getKeyAsString();Object from = bucket.getFrom();Object to = bucket.getTo();long docCount = bucket.getDocCount();System.out.println(String.format("key: %s ,from: %s ,to: %s ,docCount: %s",key,from,to,docCount));}}6.9.3 統(tǒng)計(jì)聚合
他可以幫你查詢指定field 的最大值，最小值，平均值，平方和... 使用 extended_stats #統(tǒng)計(jì)聚合查詢 extended_stats POST /sms-logs-index/sms-logs-type/_search {"aggs": {"agg": {"extended_stats": {"field": "fee"}}} } // java實(shí)現(xiàn) public void aggExtendedStats() throws IOException {// 1.創(chuàng)建requestSearchRequest request = new SearchRequest(index);request.types(type);// 2. 指定使用聚合查詢方式SearchSourceBuilder builder = new SearchSourceBuilder();builder.aggregation(AggregationBuilders.extendedStats("agg").field("fee"));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果ExtendedStats extendedStats = response.getAggregations().get("agg");System.out.println("最大值："+extendedStats.getMaxAsString()+",最小值："+extendedStats.getMinAsString());}6.9.4 其他聚合查詢 查看官方文檔
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-aggregations-metrics-weight-avg-aggregation.html
6.10 地圖經(jīng)緯度搜索
#創(chuàng)建一個(gè)經(jīng)緯度索引,指定一個(gè) name ,一個(gè)location PUT /map {"settings": {"number_of_shards": 5,"number_of_replicas": 1},"mappings": {"map":{"properties":{"name":{"type":"text"},"location":{"type":"geo_point"}}}} }#添加測(cè)試數(shù)據(jù) PUT /map/map/1 {"name":"天安門(mén)","location":{"lon": 116.403694,"lat":39.914492} }PUT /map/map/2 {"name":"百望山","location":{"lon": 116.26284,"lat":40.036576} }PUT /map/map/3 {"name":"北京動(dòng)物園","location":{"lon": 116.347352,"lat":39.947468} }6.10.1 ES 的地圖檢索方式
geo_distance :直線距離檢索方式 geo_bounding_box: 以2個(gè)點(diǎn)確定一個(gè)矩形，獲取再矩形內(nèi)的數(shù)據(jù) geo_polygon:以多個(gè)點(diǎn)，確定一個(gè)多邊形，獲取多邊形的全部數(shù)據(jù)6.10.2 基于RESTFul 實(shí)現(xiàn)地圖檢索
geo_distance
#geo_distance POST /map/map/_search {"query": {"geo_distance":{#確定一個(gè)點(diǎn)"location":{"lon":116.434739,"lat":39.909843},#確定半徑"distance":20000,#指定形狀為圓形"distance_type":"arc"}} } #geo_bounding_box POST /map/map/_search {"query":{"geo_bounding_box":{"location":{"top_left":{"lon":116.327805,"lat":39.95499},"bottom_right":{"lon": 116.363162,"lat":39.938395}}}} } #geo_polygon POST /map/map/_search {"query":{"geo_polygon":{"location":{# 指定多個(gè)點(diǎn)確定 位置"points":[{"lon":116.220296,"lat":40.075013},{"lon":116.346777,"lat":40.044751},{"lon":116.236106,"lat":39.981533} ]}}} }6.10.3 java 實(shí)現(xiàn) geo_polygon
public class GeoDemo {RestHighLevelClient client = EsClient.getClient();String index = "map";String type="map";@Testpublic void GeoPolygon() throws IOException {// 1.創(chuàng)建searchRequestSearchRequest request = new SearchRequest(index);request.types(type);// 2.指定 檢索方式SearchSourceBuilder builder = new SearchSourceBuilder();List<GeoPoint> points = new ArrayList<>();points.add(new GeoPoint(40.075013,116.220296));points.add(new GeoPoint(40.044751,116.346777));points.add(new GeoPoint(39.981533,116.236106));builder.query(QueryBuilders.geoPolygonQuery("location",points));request.source(builder);// 3.執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}} }總結(jié)
以上是生活随笔為你收集整理的听某个老师的ElasticSearch记的笔记了的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。
- 上一篇： 股票的游戏规则
- 下一篇： 苹果手机几月份最便宜_苹果手机越来越便宜