原文網(wǎng)址：ElasticSearch--性能優(yōu)化相關(guān)的配置_IT利刃出鞘的博客-CSDN博客

簡介

本文介紹ElasticSearch中性能優(yōu)化有關(guān)的配置。

相關(guān)的配置有：

內(nèi)存大小

批量寫的數(shù)量

索引數(shù)量

內(nèi)存大小

說明

ElasticSearch是個Java進(jìn)程，JVM默認(rèn)的最大內(nèi)存是1G。一般需要增大這個配置。

配置方法

配置文件：config/jvm.options

原來是：
##-Xms4g
##-Xmx4g

改為：
-Xms4g
-Xmx4g

報錯信息

如果這個值不夠大，會報的錯如下：

主要錯誤信息：

Caused by: org.springframework.data.elasticsearch.UncategorizedElasticsearchException: Elasticsearch exception [type=circuit_breaking_exception, reason=[parent] Data too large, data for [] would be [991614656/945.6mb], which is larger than the limit of [986061209/940.3mb], real usage: [991390584/945.4mb], new bytes reserved: [224072/218.8kb], usages [request=0/0b, fielddata=0/0b, in_flight_requests=224072/218.8kb, model_inference=0/0b, accounting=11298560/10.7mb]]; nested exception is ElasticsearchStatusException[Elasticsearch exception [type=circuit_breaking_exception, reason=[parent] Data too large, data for [] would be [991614656/945.6mb], which is larger than the limit of [986061209/940.3mb], real usage: [991390584/945.4mb], new bytes reserved: [224072/218.8kb], usages [request=0/0b, fielddata=0/0b, in_flight_requests=224072/218.8kb, model_inference=0/0b, accounting=11298560/10.7mb]]]

詳細(xì)錯誤信息：

Caused by: org.springframework.data.elasticsearch.UncategorizedElasticsearchException: Elasticsearch exception [type=circuit_breaking_exception, reason=[parent] Data too large, data for [] would be [991614656/945.6mb], which is larger than the limit of [986061209/940.3mb], real usage: [991390584/945.4mb], new bytes reserved: [224072/218.8kb], usages [request=0/0b, fielddata=0/0b, in_flight_requests=224072/218.8kb, model_inference=0/0b, accounting=11298560/10.7mb]]; nested exception is ElasticsearchStatusException[Elasticsearch exception [type=circuit_breaking_exception, reason=[parent] Data too large, data for [] would be [991614656/945.6mb], which is larger than the limit of [986061209/940.3mb], real usage: [991390584/945.4mb], new bytes reserved: [224072/218.8kb], usages [request=0/0b, fielddata=0/0b, in_flight_requests=224072/218.8kb, model_inference=0/0b, accounting=11298560/10.7mb]]]at org.springframework.data.elasticsearch.core.ElasticsearchExceptionTranslator.translateExceptionIfPossible(ElasticsearchExceptionTranslator.java:67)at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.translateException(ElasticsearchRestTemplate.java:370)at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.execute(ElasticsearchRestTemplate.java:353)at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.doBulkOperation(ElasticsearchRestTemplate.java:238)at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.bulkIndex(ElasticsearchRestTemplate.java:191)at org.springframework.data.elasticsearch.core.DocumentOperations.bulkIndex(DocumentOperations.java:152)at org.springframework.data.elasticsearch.core.AbstractElasticsearchTemplate.save(AbstractElasticsearchTemplate.java:175)at org.springframework.data.elasticsearch.core.AbstractElasticsearchTemplate.save(AbstractElasticsearchTemplate.java:159)

錯誤的解析

Data too large, data for [] would be [991614656/945.6mb] ? ?//A
which is larger than the limit of [986061209/940.3mb] ? ? ? //B
real usage: [991390584/945.4mb] ? ? ? ? ? ? ? ? ? ? ? ? ? ? //C
new bytes reserved: [224072/218.8kb] ? ? ? ? ? ? ? ? ? ? ? ?//D

這里有4個數(shù)值：

• A：本次將會消耗的總內(nèi)存
• B：上限，超過這個就報錯。（缺省值是ES最大內(nèi)存的95%)
• C：本機(jī)上ES進(jìn)程已使用的內(nèi)存大小
• D：你本次操作（或者說執(zhí)行當(dāng)前的任務(wù)）所需要的內(nèi)存

本次操作中，C + D = A > B，所以報錯了。

批量寫的數(shù)量

說明

提高ES的批量寫入數(shù)據(jù)的性能。

配置方法

配置文件：config/elasticsearch.yml

添加下邊一行：

thread_pool.write.queue_size: 1000

報錯信息

如果這個值不夠大，會報429錯誤：

主要錯誤信息：

{"error":{"root_cause":[{"type":"remote_transport_exception","reason":"[elasticsearch-0][10.16.9.130:9300][indices:data/write/update[s]]"}],"type":"es_rejected_execution_exception","reason":"rejected execution of org.elasticsearch.transport.TransportService$7@19a5cf02 on EsThreadPoolExecutor[name = elasticsearch-0/write, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@389297ad[Running, pool size = 2, active threads = 2, queued tasks = 200, completed tasks = 147611]]"},"status":429}

詳細(xì)錯誤信息：

Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://127.0.0.1:9200], URI [/service_instance_inventory/type/6_tcc-app-gateway-77b98ff6ff-crblx.cards_0_0/_update?refresh=true&timeout=1m], status line [HTTP/1.1 429 Too Many Requests] {"error":{"root_cause":[{"type":"remote_transport_exception","reason":"[elasticsearch-0][10.16.9.130:9300][indices:data/write/update[s]]"}],"type":"es_rejected_execution_exception","reason":"rejected execution of org.elasticsearch.transport.TransportService$7@19a5cf02 on EsThreadPoolExecutor[name = elasticsearch-0/write, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@389297ad[Running, pool size = 2, active threads = 2, queued tasks = 200, completed tasks = 147611]]"},"status":429}at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:705) ~[elasticsearch-rest-client-6.3.2.jar:6.3.2]at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235) ~[elasticsearch-rest-client-6.3.2.jar:6.3.2]at org.elasticsearch.client.RestClient.performRequest(RestClient.java:198) ~[elasticsearch-rest-client-6.3.2.jar:6.3.2]at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:522) ~[elasticsearch

索引數(shù)量

詳見：ElasticSearch--解決this action would add [5] total shards, but this cluster currently has [1000]/[1000_IT利刃出鞘的博客-CSDN博客

說明

默認(rèn)最大允許的索引數(shù)是1000個，如果業(yè)務(wù)需要大于這個數(shù)，就需要修改配置。

配置方法

法1：head工具

無需重啟ES。

PUT /_cluster/settings {"persistent": {"cluster": {"max_shards_per_node":10000}} }

persistent：永久生效，transient：臨時生效。??

法2：CURL命令

無需重啟ES。

curl --location --request PUT 'http://127.0.0.1:9200/_cluster/settings' \ --header 'Content-Type: application/json' \ --data '{"persistent":{"cluster":{"max_shards_per_node":10000}}}'

persistent：永久生效，transient：臨時生效。??

報錯信息

Elasticsearch exception [type=validation_exception, reason=Validation Failed: 1: this action would add [5] total shards, but this cluster currently has [1000]/[1000] maximum shards open;]

總結(jié)

以上是生活随笔為你收集整理的ElasticSearch--性能优化相关的配置的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。