springboot+springdata+elasticsearch+logstash+拼音分词实现全文搜索
生活随笔
收集整理的這篇文章主要介紹了
springboot+springdata+elasticsearch+logstash+拼音分词实现全文搜索
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
elasticsearch安裝
下載地址:https://www.elastic.co/downloads/elasticsearch
解壓到d盤software目錄下
CMD下:cd D:\software\elasticsearch-6.2.3\bin
執行elasticsearch.bat
訪問9200端口 出現下圖即成功
elasticsearch head插件安裝
1、安裝head插件
下載地址:https://github.com/mobz/elasticsearch-head,
下載后解壓縮。
2、安裝node.js
node下載地址nodejs.org/
$ node -vv6.8.1
npm下載地址https://www.npmjs.com
$ npm -v3.10.8
使用npm安裝grunt:
npm install –g grunt–cli
3、修改elasticsearch配置文件
編輯elasticsearch-6.2.3/config/elasticsearch.yml,加入以下內容
http.cors.enabled: truehttp.cors.allow-origin: "*"
4、修改Gruntfile.js
打開elasticsearch-head-master/Gruntfile.js,找到下面connect屬性,新增hostname: ‘0.0.0.0’:
????
connect: {
server: {
options: {
hostname: '0.0.0.0',
port: 9100,
base: '.',
keepalive: true
}
}
} 5、啟動elasticsearch-head-master 在elasticsearch-head-master/目錄下,運行啟動命令: grunt server 訪問http://localhost9100 即可開啟
開啟elasticsearch服務
集群健康值為綠色或者黃色(能用)即為成功,灰色表示未連接上elasticsearch
拼音分詞器安裝 可參考https://blog.csdn.net/napoay/article/details/53907921 pinyin分詞器的下載地址: https://github.com/medcl/elasticsearch-analysis-pinyin 下載,并用maven打包,在項目target文件夾下會生成elasticsearch-analysis-pinyin-6.2.3.zip 在elasticsearch-6.2.3\plugins目錄下新建pinyin文件夾,將elasticsearch-analysis-pinyin-6.2.3.zip解壓放入,如圖所示:
不需要其他配置,啟動elasticsearch成功即表示分詞器安裝成功。
java客戶端 采用的springboot+springdata+springcloud 集成 包結構
config package cn.search.config;
import java.net.InetAddress;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingResponse; import org.elasticsearch.client.Client; import org.elasticsearch.client.transport.TransportClient; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.transport.InetSocketTransportAddress; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.XContentFactory; import org.elasticsearch.transport.client.PreBuiltTransportClient; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration;
@Configuration public class ESConfig { @Value("${spring.data.elasticsearch.cluster-name}") private String clusterName; @Bean public Client client() throws Exception{ InetSocketTransportAddress node = new InetSocketTransportAddress( InetAddress.getByName("192.168.7.248"), 9300 ); Settings settings = Settings.builder() // .put("cluster.name",clusterName) /** * 設置client.transport.sniff為true來使客戶端去嗅探整個集群的狀態, * 把集群中其它機器的ip地址加到客戶端中,這樣做的好處是一般你不用手 * 動設置集群里所有集群的ip到連接客戶端,它會自動幫你添加,并且自動 * 發現新加入集群的機器。 */ .put("client.transport.sniff",true) .put("client.transport.ignore_cluster_name",true) .build(); TransportClient client = new PreBuiltTransportClient(settings); client.addTransportAddress(node); return client; }
}
實體類 package cn.search.entity;
import lombok.*; import org.hibernate.annotations.DynamicUpdate; import org.hibernate.annotations.Type; import org.springframework.data.annotation.Id; import org.springframework.data.annotation.LastModifiedDate; import org.springframework.data.elasticsearch.annotations.Document; import org.springframework.data.elasticsearch.annotations.Field; import org.springframework.data.elasticsearch.annotations.FieldType;
import cn.search.constant.EntityStatusConstant;
import javax.persistence.*;
import java.time.LocalDateTime; import java.util.ArrayList; import java.util.List;
@Getter @Setter //指定索引和類型 //elasticsearch中index(索引)相當于數據庫,type相當于數據庫的表, //文檔相當于數據庫的row //shards 設置分片 replicas設置備份 也可以通過put創建索引時指定 @Document(indexName="elasticsearch",type="doc")//shards=8,replicas=2 public class Program{ @Id private Long id; @Field(type = FieldType.Date) private LocalDateTime createdDate; @Field(type = FieldType.Date) private LocalDateTime modifiedDate; @Field(type = FieldType.Integer) private Integer status; @Field(type = FieldType.Integer) private Integer softDeleted= EntityStatusConstant.ENTITY_IS_DELETED_FALSE; @Field(type = FieldType.text) private String programId; @Field(type = FieldType.text) private String area; @Field(type = FieldType.text) private String description; @Field(type = FieldType.text) private String episodeNo; @Field(type = FieldType.text) private String genre; @Field(type = FieldType.text) private String language; @Field(type = FieldType.text) private String rating; @Field(type = FieldType.text) private String releaseYear; @Field(type = FieldType.text) private String season; @Field(type = FieldType.text) private String SeriesType; @Field(type = FieldType.text) private String subGenre; @Field(type = FieldType.text, searchAnalyzer="pinyin_analyzer",analyzer="pinyin_analyzer") //指定分詞器 private String title; @Field(type = FieldType.text) private String type; @Field(type = FieldType.text) private String directors; @Field(type = FieldType.text) private String writers; @Field(type = FieldType.text) private String casts; @Field(type = FieldType.text) private String channelName; @Field(type = FieldType.text) private String channelId;
}
注:elasticsearch5.x版本開始type類型沒有string,用keyword和text替代 Repository package cn.search.repository; import java.util.List; import org.springframework.data.domain.Page; import org.springframework.data.domain.Pageable; import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import cn.search.entity.Program; import cn.search.entity.ProgramBak; public interface ESRepository extends ElasticsearchRepository<Program, Long>{ //Page<Program> findByChannelName(String channelName,Pageable pageable); Page<Program> findByTitle(String title,Pageable pageable); List<Program> findByTitle(String title); }
和springJPA用法一致
service 接口: package cn.search.sercice;
import java.util.List;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Sort;
import cn.search.entity.Program;
public interface SearchService { Page<Program> getProgramBykeyWord(String keyWord,Integer offset, Integer limit, Sort sort);
List<Program> getProgramBykeyWord(String keyWord); } 實現類: package cn.search.sercice.impl;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.domain.Page; import org.springframework.data.domain.PageRequest; import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Sort; import org.springframework.stereotype.Service;
import cn.search.entity.Program; import cn.search.entity.ProgramBak; import cn.search.repository.ESRepository; import cn.search.sercice.SearchService;
@Service("SearchService") public class SearchServiceImpl implements SearchService { private ESRepository esRepository; @Autowired public SearchServiceImpl(ESRepository esRepository){ this.esRepository = esRepository; }
@Override public Page<Program> getProgramBykeyWord(String keyWord, Integer offset, Integer limit, Sort sort) { // TODO Auto-generated method stub Pageable pageable=PageRequest.of(offset/limit,limit,sort); Page<Program> page = esRepository.findByTitle(keyWord, pageable); return page; }
@Override public List<Program> getProgramBykeyWord(String keyWord) { // TODO Auto-generated method stub return esRepository.findByTitle(keyWord); }
}
controller層 package cn.search.controller;
import java.util.List; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.domain.Page; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController;
import cn.search.entity.Program; import cn.search.entity.ProgramBak; import cn.search.repository.ESRepository; import cn.search.request.ProgramPageRequest; import cn.search.sercice.SearchService;
@RestController public class ESController { private SearchService esService; @Autowired private ESRepository repository; @Autowired public ESController(SearchService esService){ this.esService = esService; } @GetMapping("/es/search") public Page<Program> findProgramByKeyWord(ProgramPageRequest pageRequest){ Page<Program> page = esService.getProgramBykeyWord(pageRequest.getKeyword(), pageRequest.getOffset(), pageRequest.getLimit(),pageRequest.getSort(pageRequest.getOrderSort(),pageRequest.getFieldConvert())); return page; } @GetMapping("/es/find") public List<Program> getProgramByKeyWord(@RequestParam(value="keyword") String keyWord){ List<Program> list = esService.getProgramBykeyWord(keyWord); return list; } @GetMapping("/es/save") public String save(){ Program p = new Program(); p.setTitle("晚間新聞"); p.setChannelName("江蘇衛視"); p.setProgramId("code"); p.setGenre("新聞"); // p.setPictureUrl("111"); // p.setThumbUrl("222"); repository.save(p); return "OK"; } }
yml文件 spring: application: name: search-service data: elasticsearch: cluster-name: elasticsearch local: false repositories: enabled: true server: port: 8080 eureka: client: service-url: defaultZone: http://register-server1/eureka/ instance: prefer-ip-address: true
pom依賴: <?xml version="1.0"?> <project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>cn.zgys</groupId> <artifactId>sichuanlive</artifactId> <version>1.0.0</version> </parent> <groupId>cn.zgys</groupId> <artifactId>search</artifactId> <version>1.0.0</version> <name>search</name>
<properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>1.8</java.version> <spring-cloud.version>Finchley.BUILD-SNAPSHOT</spring-cloud.version> </properties>
<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <scope>runtime</scope> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <version>1.16.20</version> <scope>provided</scope> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.47</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> <version>3.7</version> </dependency> </dependencies>
<dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>
<build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build>
<repositories> <repository> <id>spring-snapshots</id> <name>Spring Snapshots</name> <url>https://repo.spring.io/snapshot</url> <snapshots> <enabled>true</enabled> </snapshots> </repository> <repository> <id>spring-milestones</id> <name>Spring Milestones</name> <url>https://repo.spring.io/milestone</url> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repositories> </project>
注:如果要單獨啟動 將pom文件中的spring-cloud-starter-netflix-eureka-client依賴、yml文件的eureka-Client、啟動類上的@EnableEurekaClient注釋掉,并在@SpringBootApplication后添加(exclude= {DataSourceAutoConfiguration.class})
logstash 下載地址 https://www.elastic.co/downloads/logstash 安裝(略) sql:需要導入elasticsearch的字段
全文搜索 使用logstash從mysql同步數據到elasticsearch,由java客戶端實現全文搜索。 1、開啟elasticsearch和elasticsearch head插件 2、 在9200端口下 創建索引 發送put請求(也可以在lunix環境下使用curl命令)
{ "index" : { "analysis" : { "analyzer" : { "pinyin_analyzer" : { "tokenizer" : "my_pinyin" } }, "tokenizer" : { "my_pinyin" : { "type" : "pinyin", "keep_first_letter":true, "keep_separate_first_letter" : true, "keep_full_pinyin" : true, "keep_original" : true, "limit_first_letter_length" : 16, "lowercase" : true, "remove_duplicated_term" : true } } } } } 注: "keep_separate_first_letter" : true(根據情況選擇true或false) false:首字母搜索只有兩個首字母相同才能命中,全拼能命中 true:任何情況全拼,首字母都能命中 出現索引 點擊信息--索引信息 上面有分詞器analysis配置如下圖所示即為成功 3、啟動springboot項目
點擊信息發現 實體屬性已經全部注入,title屬性也 已經指定分詞器(“annlyzer”:“pinyin_analyzer”) 即為成功 4、開啟logstash,sql為需要導入的字段,導入數據到elasticsearch 5、測試:通過傳入的參數搜索到內容(比如傳入xw搜索到title為“新聞”,“晚間新聞”、“下午”的節目) 通過8080端口: ???
通過9200端口:
server: {
options: {
hostname: '0.0.0.0',
port: 9100,
base: '.',
keepalive: true
}
}
} 5、啟動elasticsearch-head-master 在elasticsearch-head-master/目錄下,運行啟動命令: grunt server 訪問http://localhost9100 即可開啟
開啟elasticsearch服務
集群健康值為綠色或者黃色(能用)即為成功,灰色表示未連接上elasticsearch
拼音分詞器安裝 可參考https://blog.csdn.net/napoay/article/details/53907921 pinyin分詞器的下載地址: https://github.com/medcl/elasticsearch-analysis-pinyin 下載,并用maven打包,在項目target文件夾下會生成elasticsearch-analysis-pinyin-6.2.3.zip 在elasticsearch-6.2.3\plugins目錄下新建pinyin文件夾,將elasticsearch-analysis-pinyin-6.2.3.zip解壓放入,如圖所示:
不需要其他配置,啟動elasticsearch成功即表示分詞器安裝成功。
java客戶端 采用的springboot+springdata+springcloud 集成 包結構
config package cn.search.config;
import java.net.InetAddress;
import org.elasticsearch.action.admin.indices.mapping.put.PutMappingResponse; import org.elasticsearch.client.Client; import org.elasticsearch.client.transport.TransportClient; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.transport.InetSocketTransportAddress; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.XContentFactory; import org.elasticsearch.transport.client.PreBuiltTransportClient; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration;
@Configuration public class ESConfig { @Value("${spring.data.elasticsearch.cluster-name}") private String clusterName; @Bean public Client client() throws Exception{ InetSocketTransportAddress node = new InetSocketTransportAddress( InetAddress.getByName("192.168.7.248"), 9300 ); Settings settings = Settings.builder() // .put("cluster.name",clusterName) /** * 設置client.transport.sniff為true來使客戶端去嗅探整個集群的狀態, * 把集群中其它機器的ip地址加到客戶端中,這樣做的好處是一般你不用手 * 動設置集群里所有集群的ip到連接客戶端,它會自動幫你添加,并且自動 * 發現新加入集群的機器。 */ .put("client.transport.sniff",true) .put("client.transport.ignore_cluster_name",true) .build(); TransportClient client = new PreBuiltTransportClient(settings); client.addTransportAddress(node); return client; }
}
實體類 package cn.search.entity;
import lombok.*; import org.hibernate.annotations.DynamicUpdate; import org.hibernate.annotations.Type; import org.springframework.data.annotation.Id; import org.springframework.data.annotation.LastModifiedDate; import org.springframework.data.elasticsearch.annotations.Document; import org.springframework.data.elasticsearch.annotations.Field; import org.springframework.data.elasticsearch.annotations.FieldType;
import cn.search.constant.EntityStatusConstant;
import javax.persistence.*;
import java.time.LocalDateTime; import java.util.ArrayList; import java.util.List;
@Getter @Setter //指定索引和類型 //elasticsearch中index(索引)相當于數據庫,type相當于數據庫的表, //文檔相當于數據庫的row //shards 設置分片 replicas設置備份 也可以通過put創建索引時指定 @Document(indexName="elasticsearch",type="doc")//shards=8,replicas=2 public class Program{ @Id private Long id; @Field(type = FieldType.Date) private LocalDateTime createdDate; @Field(type = FieldType.Date) private LocalDateTime modifiedDate; @Field(type = FieldType.Integer) private Integer status; @Field(type = FieldType.Integer) private Integer softDeleted= EntityStatusConstant.ENTITY_IS_DELETED_FALSE; @Field(type = FieldType.text) private String programId; @Field(type = FieldType.text) private String area; @Field(type = FieldType.text) private String description; @Field(type = FieldType.text) private String episodeNo; @Field(type = FieldType.text) private String genre; @Field(type = FieldType.text) private String language; @Field(type = FieldType.text) private String rating; @Field(type = FieldType.text) private String releaseYear; @Field(type = FieldType.text) private String season; @Field(type = FieldType.text) private String SeriesType; @Field(type = FieldType.text) private String subGenre; @Field(type = FieldType.text, searchAnalyzer="pinyin_analyzer",analyzer="pinyin_analyzer") //指定分詞器 private String title; @Field(type = FieldType.text) private String type; @Field(type = FieldType.text) private String directors; @Field(type = FieldType.text) private String writers; @Field(type = FieldType.text) private String casts; @Field(type = FieldType.text) private String channelName; @Field(type = FieldType.text) private String channelId;
}
注:elasticsearch5.x版本開始type類型沒有string,用keyword和text替代 Repository package cn.search.repository; import java.util.List; import org.springframework.data.domain.Page; import org.springframework.data.domain.Pageable; import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import cn.search.entity.Program; import cn.search.entity.ProgramBak; public interface ESRepository extends ElasticsearchRepository<Program, Long>{ //Page<Program> findByChannelName(String channelName,Pageable pageable); Page<Program> findByTitle(String title,Pageable pageable); List<Program> findByTitle(String title); }
和springJPA用法一致
service 接口: package cn.search.sercice;
import java.util.List;
import org.springframework.data.domain.Page; import org.springframework.data.domain.Sort;
import cn.search.entity.Program;
public interface SearchService { Page<Program> getProgramBykeyWord(String keyWord,Integer offset, Integer limit, Sort sort);
List<Program> getProgramBykeyWord(String keyWord); } 實現類: package cn.search.sercice.impl;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.domain.Page; import org.springframework.data.domain.PageRequest; import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Sort; import org.springframework.stereotype.Service;
import cn.search.entity.Program; import cn.search.entity.ProgramBak; import cn.search.repository.ESRepository; import cn.search.sercice.SearchService;
@Service("SearchService") public class SearchServiceImpl implements SearchService { private ESRepository esRepository; @Autowired public SearchServiceImpl(ESRepository esRepository){ this.esRepository = esRepository; }
@Override public Page<Program> getProgramBykeyWord(String keyWord, Integer offset, Integer limit, Sort sort) { // TODO Auto-generated method stub Pageable pageable=PageRequest.of(offset/limit,limit,sort); Page<Program> page = esRepository.findByTitle(keyWord, pageable); return page; }
@Override public List<Program> getProgramBykeyWord(String keyWord) { // TODO Auto-generated method stub return esRepository.findByTitle(keyWord); }
}
controller層 package cn.search.controller;
import java.util.List; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.domain.Page; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController;
import cn.search.entity.Program; import cn.search.entity.ProgramBak; import cn.search.repository.ESRepository; import cn.search.request.ProgramPageRequest; import cn.search.sercice.SearchService;
@RestController public class ESController { private SearchService esService; @Autowired private ESRepository repository; @Autowired public ESController(SearchService esService){ this.esService = esService; } @GetMapping("/es/search") public Page<Program> findProgramByKeyWord(ProgramPageRequest pageRequest){ Page<Program> page = esService.getProgramBykeyWord(pageRequest.getKeyword(), pageRequest.getOffset(), pageRequest.getLimit(),pageRequest.getSort(pageRequest.getOrderSort(),pageRequest.getFieldConvert())); return page; } @GetMapping("/es/find") public List<Program> getProgramByKeyWord(@RequestParam(value="keyword") String keyWord){ List<Program> list = esService.getProgramBykeyWord(keyWord); return list; } @GetMapping("/es/save") public String save(){ Program p = new Program(); p.setTitle("晚間新聞"); p.setChannelName("江蘇衛視"); p.setProgramId("code"); p.setGenre("新聞"); // p.setPictureUrl("111"); // p.setThumbUrl("222"); repository.save(p); return "OK"; } }
yml文件 spring: application: name: search-service data: elasticsearch: cluster-name: elasticsearch local: false repositories: enabled: true server: port: 8080 eureka: client: service-url: defaultZone: http://register-server1/eureka/ instance: prefer-ip-address: true
pom依賴: <?xml version="1.0"?> <project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <modelVersion>4.0.0</modelVersion> <parent> <groupId>cn.zgys</groupId> <artifactId>sichuanlive</artifactId> <version>1.0.0</version> </parent> <groupId>cn.zgys</groupId> <artifactId>search</artifactId> <version>1.0.0</version> <name>search</name>
<properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> <java.version>1.8</java.version> <spring-cloud.version>Finchley.BUILD-SNAPSHOT</spring-cloud.version> </properties>
<dependencies> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <scope>runtime</scope> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> </dependency> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> <version>1.16.20</version> <scope>provided</scope> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.47</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> <version>3.7</version> </dependency> </dependencies>
<dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>
<build> <plugins> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven-plugin</artifactId> </plugin> </plugins> </build>
<repositories> <repository> <id>spring-snapshots</id> <name>Spring Snapshots</name> <url>https://repo.spring.io/snapshot</url> <snapshots> <enabled>true</enabled> </snapshots> </repository> <repository> <id>spring-milestones</id> <name>Spring Milestones</name> <url>https://repo.spring.io/milestone</url> <snapshots> <enabled>false</enabled> </snapshots> </repository> </repositories> </project>
注:如果要單獨啟動 將pom文件中的spring-cloud-starter-netflix-eureka-client依賴、yml文件的eureka-Client、啟動類上的@EnableEurekaClient注釋掉,并在@SpringBootApplication后添加(exclude= {DataSourceAutoConfiguration.class})
logstash 下載地址 https://www.elastic.co/downloads/logstash 安裝(略) sql:需要導入elasticsearch的字段
全文搜索 使用logstash從mysql同步數據到elasticsearch,由java客戶端實現全文搜索。 1、開啟elasticsearch和elasticsearch head插件 2、 在9200端口下 創建索引 發送put請求(也可以在lunix環境下使用curl命令)
{ "index" : { "analysis" : { "analyzer" : { "pinyin_analyzer" : { "tokenizer" : "my_pinyin" } }, "tokenizer" : { "my_pinyin" : { "type" : "pinyin", "keep_first_letter":true, "keep_separate_first_letter" : true, "keep_full_pinyin" : true, "keep_original" : true, "limit_first_letter_length" : 16, "lowercase" : true, "remove_duplicated_term" : true } } } } } 注: "keep_separate_first_letter" : true(根據情況選擇true或false) false:首字母搜索只有兩個首字母相同才能命中,全拼能命中 true:任何情況全拼,首字母都能命中 出現索引 點擊信息--索引信息 上面有分詞器analysis配置如下圖所示即為成功 3、啟動springboot項目
點擊信息發現 實體屬性已經全部注入,title屬性也 已經指定分詞器(“annlyzer”:“pinyin_analyzer”) 即為成功 4、開啟logstash,sql為需要導入的字段,導入數據到elasticsearch 5、測試:通過傳入的參數搜索到內容(比如傳入xw搜索到title為“新聞”,“晚間新聞”、“下午”的節目) 通過8080端口: ???
通過9200端口:
總結
以上是生活随笔為你收集整理的springboot+springdata+elasticsearch+logstash+拼音分词实现全文搜索的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: 如何使用sci-hub批量下载论文
- 下一篇: 材质自发光过曝之后变白的问题