該樓層疑似違規已被系統折疊?隱藏此樓查看此樓

#me the components on this agent

a1.sources = r1

a1.sinks = k1

a1.channels = c1

# Describe/configure the source

a1.sources.r1.type = exec

a1.sources.r1.command = tail -F /usr/local/test1.txt

a1.sources.r1.fileHeader = true

a1.sources.r1.deserializer.outputCharset=UTF-8

# Describe the sink

a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink

a1.sinks.k1.brokerList = c100:9092,c200:9092,c300:9092

a1.sinks.k1.topic = test5

a1.sinks.k1.requiredAcks = 1

a1.sinks.k1.batchSize = 20

a1.sinks.k1.channel = memcnl

#a1.sinks.k1.type = avro

#a1.sinks.k1.hostname = 192.168.28.134

#a1.sinks.k1.port = 44444

# Use a channel which buffers events in memory

a1.channels.c1.type = file

a1.channels.c1.checkpointDir = /usr/local/chk

a1.channels.c1.dataDirs = /usr/local/flumeData

a1.channels.c1.checkpointOnClose = true

# Bind the source and sink to the channel

a1.sources.r1.channels = c1

a1.sinks.k1.channel = c1

把flume停掉，重啟后，數據會重復，怎么解決呢

創作挑戰賽新人創作獎勵來咯，堅持創作打卡瓜分現金大獎

總結

以上是生活随笔為你收集整理的flume数据丢失与重复_Flume监听文件到kafka，文件通道，kafka消费重复问题的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。