Apache seaTunnel的本地启动
準(zhǔn)備spark環(huán)境:
linux啟動Spark本地模式(Local模式)_怪只怪滿眼盡是人間煙火-CSDN博客首先安裝java環(huán)境下載spark:wget https://archive.apache.org/dist/spark/spark-2.3.0/spark-2.3.0-bin-hadoop2.7.tgz創(chuàng)建解析文件:mkdir /opt/inputcd /opt/input/touch wenjian.txt文件中輸入測試內(nèi)容:hello azhello spark金庸防火墻:CentOS7systemctl stop firewalldCentOShttps://qushen.blog.csdn.net/article/details/122997521
下載 seatunnel:
wget https://github.com/InterestingLab/seatunnel/releases/download/v1.5.7/seatunnel-1.5.7.zip -O seatunnel-1.5.7.zip解壓,并采用軟連接重命名:
unzip seatunnel-1.5.7.zip ln -s seatunnel-1.5.7.zip seatunnel編輯 config/seatunnel-env.sh, 指定必須環(huán)境配置如SPARK_HOME
編輯 config/application.conf, 它決定了seatunnel啟動后,數(shù)據(jù)輸入,處理,輸出的方式和邏輯。
spark {# seatunnel defined streaming batch duration in secondsspark.streaming.batchDuration = 5spark.app.name = "seatunnel"spark.ui.port = 13000 }input {socketStream {} }filter {split {fields = ["msg", "name"]delimiter = ","} }output {stdout {} }重啟啟動一個窗口,啟動netcat server用于發(fā)送數(shù)據(jù),輸入:
nc -l -p 9999啟動seatunnel
./bin/start-seatunnel.sh --master local[4] --deploy-mode client --config ./config/application.conf?然后,切換到netcat server那個窗口,發(fā)送數(shù)據(jù):
Hello World, Garyseatunnel日志打印出:
總結(jié)
以上是生活随笔為你收集整理的Apache seaTunnel的本地启动的全部內(nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: linux启动Spark本地模式(Loc
- 下一篇: ClickHouse分布式集群部署