使用G1后報(bào)錯(cuò)

Caused by: org.elasticsearch.common.breaker.CircuitBreakingException: [parent] Data too large, data for [<transport_request>]

ES: 7.5.0

今天在優(yōu)化ES的GC配置的時(shí)候又踩了坑，因?yàn)橹霸?.1上使用過(guò)G1，感覺(jué)效果還不錯(cuò)，這次的7.5上也就直接升級(jí)成了G1，jvm.options的配置是

-Xms24g -Xmx24g## GC configuration -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly## G1GC Configuration # NOTE: G1GC is only supported on JDK version 10 or later. # To use G1GC uncomment the lines below. 10-:-XX:-UseConcMarkSweepGC 10-:-XX:-UseCMSInitiatingOccupancyOnly 10-:-XX:+UseG1GC 10-:-XX:InitiatingHeapOccupancyPercent=75

升級(jí)完成之后，集群中的節(jié)點(diǎn)在查詢(xún)的時(shí)候有時(shí)候會(huì)報(bào)錯(cuò)

Caused by: org.elasticsearch.common.breaker.CircuitBreakingException: [parent] Data too large, data for [<transport_request>] would be [24714355312/23gb], which is larger than the limit of [24481313587/22.7gb], real usage: [24714353312/23gb], new bytes reserved: [2000/1.9kb], usages [request=0/0b, fielddata=11807/11.5kb, in_flight_requests=2932/2.8kb, accounting=45086480/42.9mb]at org.elasticsearch.indices.breaker.HierarchyCircuitBreakerService.checkParentLimit(HierarchyCircuitBreakerService.java:343) ~[elasticsearch-7.5.0.jar:7.5.0]at org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker.addEstimateBytesAndMaybeBreak(ChildMemoryCircuitBreaker.java:128) ~[elasticsearch-7.5.0.jar:7.5.0]

從這個(gè)報(bào)錯(cuò)來(lái)看是觸發(fā)了集群的熔斷操作導(dǎo)致的，最開(kāi)始沒(méi)有注意，后來(lái)發(fā)現(xiàn)在低流量的時(shí)候也很容易觸發(fā)復(fù)現(xiàn)，感覺(jué)問(wèn)題似乎并不簡(jiǎn)單。使用關(guān)鍵字看到官方有這樣的文檔

G1 GC were setup to use an InitiatingHeapOccupancyPercent of 75. This could leave used memory at a very high level for an extended duration, triggering the real memory circuit breaker even at low activity levels. The value is a threshold for old generation usage relative to total heap size and thus it should leave room for the new generation. Default in G1 is to allow up to 60 percent for new generation and this could mean that the threshold was effectively at 135% heap usage. GC would still kick in of course and eventually enough mixed collections would take place such that adaptive adjustment of IHOP kicks in.The JVM has adaptive setting of the IHOP, but this does not kick in until it has sampled a few collections. A newly started, relatively quiet server with primarily new generation activity could thus experience heap above 95% frequently for a duration.The changes here are two-fold:Use 30% default for IHOP (the JVM default of 45 could still mean 105% heap usage threshold and did not fully ensure not to hit the circuit breaker with low activity) Set G1ReservePercent=25. This is used by the adaptive IHOP mechanism, meaning old/mixed GC should kick in no later than at 75% heap. This ensures IHOP stays compatible with the real memory circuit breaker also after being adjusted by adaptive IHOP.

也就是說(shuō)

10-:-XX:InitiatingHeapOccupancyPercent=75

這個(gè)配置是有問(wèn)題的，這個(gè)配置了老年代的初始化內(nèi)存，但是G1允許新生代使用60%的jvm內(nèi)存，所以總的內(nèi)存量可能很容易達(dá)到135%,因?yàn)檫@個(gè)時(shí)候可能并沒(méi)有占滿(mǎn)內(nèi)存，G1的回收可能不會(huì)觸發(fā)，但是對(duì)于ES來(lái)說(shuō)，已經(jīng)觸發(fā)了內(nèi)存熔斷機(jī)制，所以即使請(qǐng)求量級(jí)不大，可能也會(huì)頻繁報(bào)內(nèi)存熔斷的錯(cuò)誤，需要注意哦。

修正的方法是

10-:-XX:-UseConcMarkSweepGC 10-:-XX:-UseCMSInitiatingOccupancyOnly 10-:-XX:+UseG1GC 10-:-XX:G1ReservePercent=25 10-:-XX:InitiatingHeapOccupancyPercent=30

增加和修改了配置

10-:-XX:G1ReservePercent=25 10-:-XX:InitiatingHeapOccupancyPercent=30

坑死了，剛重啟完畢，又要重啟一遍😭

總結(jié)

以上是生活随笔為你收集整理的使用G1后报错-CircuitBreakingException: [parent] Data too large的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。