flume搭建
Flume單機(jī)版環(huán)境搭建
搭建步驟：
1.環(huán)境準(zhǔn)備：在win7系統(tǒng)? 內(nèi)存（8G）的硬件環(huán)境下，搭建1臺(tái)centOS虛擬機(jī)
1.1給機(jī)器配置主機(jī)名（ip需要固定）
1.2 Xshell遠(yuǎn)程命令連接上該虛擬機(jī)
1.3 為該臺(tái)虛擬機(jī)配置用戶/密碼登錄
2.配置jdk
2.1指定目錄下解壓jdk壓縮包?? # /usr/software/
2.2 配置jdk環(huán)境變量
?? JAVA_HOME=/usr/software/jdk1.8.0_131
PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/sbin
3.配置flume
3.1指定目錄下解壓apache-flume-1.8.0-bin壓縮包?? # /usr/software/
3.2 復(fù)制apache-flume-1.8.0-bin/conf/flume-conf.properties.template文件至apache-flume-1.8.0-bin/http.conf，修改http.conf
???? 編寫配置文件（小案例）
??? ???? ＃example.conf：單節(jié)點(diǎn)Flume配置
??? ???? ＃命名Agent a1的組件
???????? a1.sources? =? r1
???????? a1.sinks? =? k1
???????? a1.channels? =? c1
??? ???? ＃描述/配置Source
???????? a1.sources.r1.type? =? netcat
???????? a1.sources.r1.bind? =? 0.0.0.0
???????? a1.sources.r1.port? =? 44444
?
??? ??? ＃描述Sink
???????? a1.sinks.k1.type? =? logger
??? ??? ＃描述內(nèi)存Channel
???????? a1.channels.c1.type? =? memory
???????? a1.channels.c1.capacity? =? 1000
???????? a1.channels.c1.transactionCapacity? =? 100
??? ???? ＃為Channle綁定Source和Sink
???????? a1.sources.r1.channels? =? c1
???????? a1.sinks.k1.channel =? c1
3.3 agent里三個(gè)組件并非耦合性較大，source必須存在，channel和sink也可以只配置其一，可直接從channel配置kafka，如下案例：
#Agent
a1.sources = r1
a1.channels = c1
#source
a1.sources.r1.type = spooldir
a1.sources.r1.spoolDir=/usr/files
a1.sources.r1.channels = c1
#channel
a1.channels.c1.type = org.apache.flume.channel.kafka.KafkaChannel
a1.channels.c1.kafka.bootstrap.servers = kafka01:9092,kafka02:9092,kafka03:9092
a1.channels.c1.kafka.topic = flume33
4.啟動(dòng)flume
在apache-flume-1.8.0-bin/目錄下啟動(dòng)
../bin/flume-ng agent --conf conf --conf-file flume-conf.properties --name a1 -Dflume.root.logger=INFO,console
?
?
???
?
轉(zhuǎn)載于:https://www.cnblogs.com/runnerjack/p/7846753.html
總結(jié)
- 上一篇： 测试工具大全
- 下一篇： 微信小程序：开发之前要知道的三件事