之前記錄的：Flink 1.9 CDH 6.3 集成
有些下載鏈接可能被官方關(guān)閉了，這里介紹1.12版本集成，并把安裝包下載地址換為百度網(wǎng)盤鏈接

下載安裝包

鏈接: https://pan.baidu.com/s/112fiaaMAMMXMsyiTDh3qjg 提取碼: ar5f

安裝包內(nèi)容

FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel.sha FLINK_ON_YARN-1.12.1.jar manifest.json

安裝

第一步

將FLINK_ON_YARN-1.12.1.jar放到 /opt/cloudera/csd目錄下

第二步

在我們安裝httpd服務(wù) /var/www/html目錄下創(chuàng)建flink目錄，將下面三個文件放在flink目錄下

FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel.sha manifest.json

第三步 修改httpd服務(wù)

參考：https://datamining.blog.csdn.net/article/details/121630806
新增 .parcel配置

[root@localhost bigdata]# vim /etc/httpd/conf/httpd.conf#在<IfModule mime_module>模塊添加 parcel選項<IfModule mime_module>## TypesConfig points to the file containing the list of mappings from# filename extension to MIME-type.#TypesConfig /etc/mime.types## AddType allows you to add to or override the MIME configuration# file specified in TypesConfig for specific file types.##AddType application/x-gzip .tgz## AddEncoding allows you to have certain browsers uncompress# information on the fly. Note: Not all browsers support this.##AddEncoding x-compress .Z#AddEncoding x-gzip .gz .tgz## If the AddEncoding directives above are commented-out, then you# probably should define those extensions to indicate media types:#AddType application/x-compress .ZAddType application/x-gzip .gz .tgz .parcel #此處添加.parcel#

重啟http服務(wù)

[root@localhost bigdata]#systemctl restart httpd

3.配置CDH Flink Parcel

點擊配置進(jìn)入，新增flink鏈接

確認(rèn)后會在parcel中出現(xiàn)flink，如下圖
PS:這里截圖是之前版本的

點擊下載

點擊分配

點擊激活

激活成功

重啟服務(wù)

systemctl restart cloudera-scm-server

添加服務(wù)

添加Flink

選擇需要安裝節(jié)點點擊繼續(xù)
注意這里安裝一個節(jié)點就行，yarn模式都是在yarn中運行

啟動成功

重啟Cloudera Management Service，(不重啟的話CDH無法監(jiān)控到)

問題解決

NoClassDefFoundError: org/apache

啟動flink時失敗，日志報錯招不到Y(jié)arn的jar包

下載jar包

https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-hadoop-3-uber/3.1.1.7.2.9.0-173-9.0

上傳到目錄

/opt/cloudera/parcels/FLINK/lib/flink/lib

java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;

16:12:34.454 [main] ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli - Error while running the Flink session. java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;at org.apache.flink.yarn.cli.FlinkYarnSessionCli.<init>(FlinkYarnSessionCli.java:230) ~[flink-dist_2.12-1.12.1.jar:1.12.1]at org.apache.flink.yarn.cli.FlinkYarnSessionCli.<init>(FlinkYarnSessionCli.java:156) ~[flink-dist_2.12-1.12.1.jar:1.12.1]at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:851) [flink-dist_2.12-1.12.1.jar:1.12.1]------------------------------------------------------------The program finished with the following exception:java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;at org.apache.flink.yarn.cli.FlinkYarnSessionCli.<init>(FlinkYarnSessionCli.java:230)at org.apache.flink.yarn.cli.FlinkYarnSessionCli.<init>(FlinkYarnSessionCli.java:156)at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:851) [root@r-wb-14 ~]#

下載jar包放到目錄

https://mvnrepository.com/artifact/commons-cli/commons-cli/1.3.1 /opt/cloudera/parcels/FLINK/lib/flink/lib

kerberos login configuration is invalid

原因是因為配置了kerberos，上面安裝包默認(rèn)的配置，這個要刪除掉

相關(guān)資料文檔：

Flink1.12.3+CDH6.3.1集成.docx
鏈接: https://pan.baidu.com/s/1QaaihgJM7QDO7yRzi4xbCg
提取碼: 97n2*

https://blog.csdn.net/weixin_43215250/article/details/105725330

總結(jié)

以上是生活随笔為你收集整理的Flink 1.12 CDH 6.3 集成的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。