Kafka Producer重试参数retries设置取舍的示例分析
Kafka Producer重試參數retries設置取舍的示例分析,很多新手對此不是很清楚,為了幫助大家解決這個難題,下面小編將為大家詳細講解,有這方面需求的人可以來學習下,希望你能有所收獲。
1.retries參數說明
參數的設置通常是一種取舍,看下retries參數在版本0.11.3說明:
Setting a value greater than zero will cause the client to resend
any record whose send fails with a potentially transient error.
Note that this retry is no different than if the client resent the
record upon receiving the error.
Allowing retries without setting max.in.flight.requests.per.connection to 1 will potentially change
the ordering of records because if two batches are sent to a single
partition, and the first fails and is retried but the second succeeds,
then the records in the second batch may appear first.
備注:當發送失敗時客戶端會進行重試,重試的次數由retries指定,此參數默認設置為0。即:快速失敗模式,當發送失敗時由客戶端來處理后續是否要進行繼續發送。如果設置retries大于0而沒有設置max.in.flight.requests.per.connection=1則意味著放棄發送消息的順序性。
2.retries使用建議
使用retries的默認值交給使用方自己去控制,結果往往是不處理。所以通用設置建議設置如下:
retries = Integer.MAX_VALUE
max.in.flight.requests.per.connection = 1
備注:這樣設置后,發送客戶端會一直進行重試直到broker返回ack;同時只有一個連接向broker發送數據保證了數據的順序性。在Leader選舉、集群中一個broker掛掉時,發送端會一直重試直到Leader選舉結束。避免由于客戶端對異常未處理造成的數據丟失,例如:遇到類似“This server is not the leader for that topic-partition”會自動恢復。
3.retries后續發展
該參數的設置已經在kafka 2.4版本中默認設置為Integer.MAX_VALUE;同時增加了delivery.timeout.ms的參數設置。
The default value for the producer's retries config was changed to
Integer.MAX_VALUE, as we introduced delivery.timeout.ms in KIP-91,
which sets an upper bound on the total time between sending a
record and receiving acknowledgement from the broker.
By default, the delivery timeout is set to 2 minutes.
KIP-91: https://cwiki.apache.org/confluence/display/KAFKA/KIP-91+Provide+Intuitive+User+Timeouts+in+The+Producer
總結
以上是生活随笔為你收集整理的Kafka Producer重试参数retries设置取舍的示例分析的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: poj1942(求组合数)
- 下一篇: 董明珠1人1套房承诺不变 哪些格力员工能