當(dāng)前位置：首頁(yè) > 编程资源 > 编程问答 >内容正文

编程问答

CDH6报错解决全记录

發(fā)布時(shí)間：2025/3/11 编程问答 22 豆豆

生活随笔收集整理的這篇文章主要介紹了 CDH6报错解决全记录小編覺(jué)得挺不錯(cuò)的,現(xiàn)在分享給大家,幫大家做個(gè)參考.

一、服務(wù)器準(zhǔn)備

1、服務(wù)器配置

準(zhǔn)備3臺(tái)服務(wù)器，建議最低配置如下（阿里云大約每臺(tái)每月六百多）：
centos7.4 8c 16G

39.101.192.109 node001
39.101.179.3 node002
39.99.236.205 node003

為服務(wù)器開(kāi)啟外網(wǎng)端口訪問(wèn)
7180、8900

2、下載安裝包

把下面列出的共12個(gè)文件下載到一臺(tái)文件服務(wù)器上

下載cdh文件到/var/www/html/cloudera-repos/目錄
（1）https://archive.cloudera.com/cdh6/6.0.0/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel
（2）https://archive.cloudera.com/cdh6/6.0.0/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256
（3）https://archive.cloudera.com/cdh6/6.0.0/parcels/manifest.json

下載repo支持文件到/var/www/html/cloudera-repos/repodata/目錄
（4）https://archive.cloudera.com/cdh6/6.3.2/redhat7/yum/repodata/repomd.xml
（5）https://archive.cloudera.com/cdh6/6.3.2/redhat7/yum/repodata/3224703272392229e02b46c2ef95286e92cce148a07c04752bcfe98870bfd675-primary.sqlite.bz2
（6）https://archive.cloudera.com/cdh6/6.3.2/redhat7/yum/repodata/4f1d5a908a6b4fd79988109121695168a65ef0846f178bbf32dc99f92c7cef36-filelists.sqlite.bz2

下載cm文件到/var/www/html/cloudera-repos/RPMS/x86_64/目錄
（7）https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-6.0.0-530873.el7.x86_64.rpm
（8）https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-6.0.0-530873.el7.x86_64.rpm
（9）https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-6.0.0-530873.el7.x86_64.rpm
（10）https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-db-2-6.0.0-530873.el7.x86_64.rpm
（11）https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPMS/x86_64/oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm

下載文件到/var/www/html/cloudera-repos/目錄
（12）https://archive.cloudera.com/cm6/6.0.0/allkeys.asc

3、配置文件服務(wù)器

啟動(dòng)一個(gè)http服務(wù)器用于安裝rpm安裝包，以及方便后面cdh安裝

cd /var/www/html/ cp ./cloudera-repos/CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256 ./cloudera-repos/CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha cp ./cloudera-repos/CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha256 ./cloudera-repos/CDH-6.0.0-1.cdh6.0.0.p0.537114-el7.parcel.sha1 python -m SimpleHTTPServer 8900

為所有需要升級(jí)的服務(wù)器配置本地repo庫(kù)

vi /etc/yum.repos.d/cloudera-manager.repo

[cloudera-manager] name=Cloudera Manager 6.0.0 baseurl=http://node001:8900/cloudera-repos/ gpgcheck=0 enabled=1

測(cè)試配置是否生效

yum deplist cloudera-manager-agent

二、環(huán)境準(zhǔn)備

1、設(shè)置服務(wù)器超時(shí)時(shí)間1小時(shí)

/etc/profile中添加

export TMOUT=3600 # 超時(shí)時(shí)間1小時(shí)

讓配置生效

source /etc/profile

/etc/ssh/sshd_config中添加

ClientAliveInterval 60 # 每60秒檢測(cè)一次客戶端是否存在 ClientAliveCountMax 3 # 檢測(cè)3次

修改配置后，重啟sshd服務(wù)

systemctl restart sshd

2、配置3節(jié)點(diǎn)免密登錄

在每一臺(tái)服務(wù)器中的/etc/hosts添加節(jié)點(diǎn)名字

172.26.53.202 node001 172.26.53.203 node002 172.26.53.204 node003

在每一臺(tái)服務(wù)器中生成rsa**，一路按回車即可

ssh-****** -t rsa

在每一臺(tái)服務(wù)器執(zhí)行如下命令，讓3臺(tái)服務(wù)器可互相免密登錄

ssh-copy-id -i ~/.ssh/id_rsa.pub root@node001 ssh-copy-id -i ~/.ssh/id_rsa.pub root@node002 ssh-copy-id -i ~/.ssh/id_rsa.pub root@node003

3、修改transparent_hugepage參數(shù)(透明大頁(yè)面壓縮)

echo never > /sys/kernel/mm/transparent_hugepage/defrag echo never > /sys/kernel/mm/transparent_hugepage/enabled echo 'echo never > /sys/kernel/mm/transparent_hugepage/defrag' >> /etc/rc.d/rc.local echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.d/rc.local

三、cdh安裝

1、每臺(tái)服務(wù)器安裝java

yum install oracle-j2sdk1.8-1.8.0+update141-1.x86_64.rpm

在/etc/profile中追加如下配置：

JAVA_HOME=/usr/java/jdk1.8.0_141-cloudera CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar PATH=$PATH:$JAVA_HOME/bin:$HOME/bin:$HOME/.local/bin

讓配置生效

source /etc/profile

2、安裝mysql

下載安裝包，安裝mysql-server

wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm rpm -ivh mysql-community-release-el7-5.noarch.rpm yum update yum install mysql-server

設(shè)置mysql大小寫不敏感（非常重要）
修改/etc/my.conf文件中 [mysqld] 的內(nèi)容

lower_case_table_names=1

啟動(dòng)mysql服務(wù)，并把mysql加入開(kāi)機(jī)啟動(dòng)

systemctl start mysqld systemctl enable mysqld

登錄到mysql

mysql -uroot -p

為root用戶授外網(wǎng)訪問(wèn)權(quán)限

GRANT ALL ON *.* TO 'root'@'%' IDENTIFIED BY '##你的密碼##'; flush privileges;

4、安裝

sudo yum clean all sudo yum upgrade cloudera-manager-daemons cloudera-manager-agent rpm -qa 'cloudera-manager-*'

5、初始化數(shù)據(jù)庫(kù)

mysql -uroot -p

# Cloudera Manager的數(shù)據(jù)庫(kù) CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm@123'; # Activity Monitor的數(shù)據(jù)庫(kù) CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon@123'; # Reports Manager的數(shù)據(jù)庫(kù) CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman@123'; # Hue的數(shù)據(jù)庫(kù) CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue@123'; # Metastore的數(shù)據(jù)庫(kù) CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON metastore.* TO 'metastore'@'%' IDENTIFIED BY 'metastore@123'; # Sentry的數(shù)據(jù)庫(kù) CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'sentry@123'; # Cloudera Navigator Audit Server的數(shù)據(jù)庫(kù) CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav@123'; # Cloudera Navigator Metadata Server的數(shù)據(jù)庫(kù) CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms@123'; # Oozie的數(shù)據(jù)庫(kù) CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie@123'; # Hive的數(shù)據(jù)庫(kù) CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci; GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'hive@123'; flush privileges;

6、使用scm初始化mysql

sh /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm

7、主節(jié)點(diǎn)啟動(dòng)server

systemctl start cloudera-scm-server systemctl enable cloudera-scm-server tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log systemctl status cloudera-scm-server

8、子節(jié)點(diǎn)啟動(dòng)agent

配置server服務(wù)器地址

vim /etc/cloudera-scm-agent/config.ini

server_host=node001

啟動(dòng)agent服務(wù)

systemctl start cloudera-scm-agent systemctl enable cloudera-scm-agent tail -f /var/log/cloudera-scm-agent/cloudera-scm-agent.log systemctl status cloudera-scm-agent

9、網(wǎng)頁(yè)訪問(wèn)cm

http://39.101.192.109:7180/

四、異常解決方案

1、SqlExceptionHelper - Table ‘scm.CM_VERSION’ doesn’t exist

如果為linux系統(tǒng)，mysql修改為大小寫不敏感，修改/etc/my.conf文件中 [mysqld] 的內(nèi)容

lower_case_table_names=1

systemctl restart mysql

刪除scm數(shù)據(jù)庫(kù)，新建數(shù)據(jù)庫(kù)，重新執(zhí)行scm_prepare_database.sh

sh /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm

2、cm-server[38046]: ERROR StatusLogger No log4j2 configuration file found.

看一下/var/log/cloudera-scm-server/cloudera-scm-server.log日志，可能為其他包中報(bào)了其他的異常，導(dǎo)致找不到log4j2的配置。例如找不到數(shù)據(jù)表，或者數(shù)據(jù)表數(shù)據(jù)為空等。

3、scm.hosts not exist 或者表無(wú)數(shù)據(jù)

（1）如果為linux系統(tǒng)，mysql修改為大小寫不敏感，刪除scm數(shù)據(jù)庫(kù)后，新建數(shù)據(jù)庫(kù)，重新執(zhí)行scm_prepare_database.sh
（2）主機(jī)名沖突：?https://blog.csdn.net/NeverGiveup54/article/details/102836562
（3）未設(shè)置主機(jī)名，在hosts中未設(shè)置主機(jī)名與ip的映射關(guān)系

4、cm安裝完成后，無(wú)法訪問(wèn)

（1）如果是自建服務(wù)器，可以關(guān)閉防火墻，或者開(kāi)放網(wǎng)絡(luò)端口
（2）如果是從阿里云或騰訊云購(gòu)買的服務(wù)器，通過(guò)管理控制臺(tái)開(kāi)放網(wǎng)絡(luò)端口

5、找不到哈希文件

sha1文件重命名為sha，要保證這里的sha的哈希在mainfest.json中一致。
在界面中修改“更多選項(xiàng)”中的列表，添加一個(gè)新的，然后再刪除，相當(dāng)于刷新一遍緩存。

6、找不到cloudera-repos/allkeys.asc

未下載這個(gè)文件，在網(wǎng)上下載這個(gè)文件后放到/var/www/html/cloudera-repos/目錄

7、cdh離線安裝無(wú)法復(fù)制文件

sha256文件重命名為sha和sha1，要保證這里的sha的哈希在mainfest.json中一致。

8、主機(jī)運(yùn)行狀態(tài)不良 / Cluster not found

https://blog.csdn.net/u011031430/article/details/74057601
在cm界面中，退回上一步

rm -f /opt/cloudera-manager/cm/lib/cloudera-scm-agent/cm_guid

或者

rm -f /var/lib/cloudera-scm-agent/cm_guid systemctl restart cloudera-scm-agent

9、CDH HDFS：NameNode is not formatted.

未使用的集群使用：hadoop namenode -format
已有數(shù)據(jù)的集群：https://blog.csdn.net/weixin_44202548/article/details/86496440

10、Starting cloudera-scm-server: [FAILED]

systemctl status -l cloudera-scm-server

查看具體的失敗原因，如果未找到可以在日志文件中查看原因

vi /var/log/cloudera-scm-server/cloudera-scm-server.log

11、cloudera-scm-agent日志中有錯(cuò)誤

rm -f /opt/cloudera-manager/cm-5.11.1/lib/cloudera-scm-agent/cm_guid

或者

rm -f /var/lib/cloudera-scm-agent/cm_guid systemctl restart cloudera-scm-agent

12、啟動(dòng)cloudera-scm-agent,cloudera-scm-server時(shí)，說(shuō)JDK不支持，要使用oracle jdk

一般在安裝前就會(huì)報(bào)這個(gè)錯(cuò)，如果你已經(jīng)安裝了oracle jdk，檢查環(huán)境變量是否配置JAVA，或者刪除系統(tǒng)中的openjdk，重新安裝oracle jdk

13、無(wú)法發(fā)出查詢：未能連接到Host Monitor

未初始化scm數(shù)據(jù)導(dǎo)致的，可能是由于在新建集群后，中斷安裝導(dǎo)致的
解決辦法為：在CM界面刪除集群，停止cm服務(wù)，并在后臺(tái)停止服務(wù)

systemctl stop cloudera-scm-agent systemctl stop cloudera-scm-server

重新從第三章的第6小結(jié)開(kāi)始

14、Hive啟動(dòng)失敗，報(bào)如下錯(cuò)誤

Command failed to run because this role has an invalid configuration. Review and correct its configuration. First error: Enable Stored Notifications in Database is required by Sentry and must be enabled.

需要在Sentry中設(shè)置hive_enable_db_notification，這個(gè)選項(xiàng)打上勾。

15、Hive啟動(dòng)失敗，報(bào)如下錯(cuò)誤

Command failed to run because server kudu has an invalid configuration. Review and correct its configuration. First error: Sentry authraizion in Kudu required that Hive is configured with Kudu.

重啟Hive后，Resume。

16、全部安裝好后，impala一直在重啟，看日志顯示：

Could not connect to meta store using any of the URIs provided. Most recent failure:.......Commection refused Failed to connect to the MetaStore Server... Failed to connect to Hive MetaStore. Retrying.

（1）重啟Hive、MetaStore，通過(guò)命令行連接Hive，看是否正常
（2）檢查hive連接的數(shù)據(jù)庫(kù)，數(shù)據(jù)庫(kù)名可能為hive或者metastore，看里面是否有29張表，表中是否有數(shù)據(jù)
（3）如果庫(kù)中沒(méi)有表或沒(méi)有數(shù)據(jù)，則

17、用beeline連接hiveServer2時(shí)，報(bào)下面的錯(cuò)

FAILED: InvalidConfigurationException hive.server2.authentication can’t be none in non-testing mode
修改hive-site.xml中的testing.mode為true

<property><name>sentry.hive.testing.mode</name><value>true</value> </property>

18、通過(guò)beeline連接hive后切換database，顯示如下：

User Hive does not have privileges for SWITCHDATABASE

新建角色，賦權(quán)限給hive用戶
https://blog.csdn.net/lvtula/article/details/89177670

19、Sentry啟動(dòng)HA的時(shí)候，報(bào)錯(cuò)，后臺(tái)提示com.mysql.jdbc.Driver找不到

在HA的節(jié)點(diǎn)中，看一下/share/lib/有沒(méi)有jdbc的jar包。如果沒(méi)有，下載一個(gè)到該目錄
https://dev.mysql.com/downloads/connector/j/5.1.html

20、HDFS、YARN webui無(wú)法訪問(wèn)

（1）如果域名綁定的是內(nèi)網(wǎng)的IP，那這個(gè)WebUI默認(rèn)綁定內(nèi)網(wǎng)IP，可以在HDFS / YARN 配置中搜索“通配符”，把NameNode和DataNode使用通配符的選項(xiàng)勾選
（2）搜索webui，關(guān)閉kerberos的配置

總結(jié)

以上是生活随笔為你收集整理的CDH6报错解决全记录的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。

上一篇：二、WIN10 64位下Pycharm打
下一篇：单片机Proteus7.8仿真和Prot