前置條件

ntp服務安裝

防火墻關閉

執(zhí)行以下優(yōu)化代碼

systemctl stop firewalld.service echo "* soft nofile 128000" >>/etc/security/limits.conf echo "* hard nofile 128000" >>/etc/security/limits.conf echo "* soft nproc 128000" >>/etc/security/limits.conf echo "* hard nproc 128000" >>/etc/security/limits.conf cat /etc/security/limits.conf sed -i 's/4096/unlimited/' /etc/security/limits.d/20-nproc.conf cat /etc/security/limits.d/20-nproc.conf sed -i 's/4096/unlimited/' /etc/security/limits.d/90-nproc.conf cat /etc/security/limits.d/90-nproc.conf ulimit -SHn 128000 ulimit -SHu 128000echo "vm.swappiness=10" >>/etc/sysctl.conf sysctl -p sysctl -a|grep swappiness echo "net.core.somaxconn = 32768 " >>/etc/sysctl.conf sysctl -p sysctl -a|grep somaxconn echo never > /sys/kernel/mm/transparent_hugepage/defrag echo never > /sys/kernel/mm/transparent_hugepage/enabled

?

CDH6.2 下載信息

https://www.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_62_download.html yum安裝：https://archive.cloudera.com/cdh6/6.2.0/redhat7/yum/ parcel安裝：https://archive.cloudera.com/cdh6/6.2.0/parcels/

?

?

?

這里我們講解使用 parcel 方式安裝

?

ClouderaManager 安裝包下載

https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-6.2.0-968826.el7.x86_64.rpm https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-6.2.0-968826.el7.x86_64.rpm https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-6.2.0-968826.el7.x86_64.rpm https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPMS/x86_64/cloudera-manager-server-db-2-6.2.0-968826.el7.x86_64.rpm https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPMS/x86_64/enterprise-debuginfo-6.2.0-968826.el7.x86_64.rpm https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPMS/x86_64/oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm https://archive.cloudera.com/cm6/6.2.0/allkeys.asc

CDH 安裝包下載

https://archive.cloudera.com/cdh6/6.2.0/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel https://archive.cloudera.com/cdh6/6.2.0/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha1 https://archive.cloudera.com/cdh6/6.2.0/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373-el7.parcel.sha256 https://archive.cloudera.com/cdh6/6.2.0/parcels/manifest.json

安裝 createrepo

作用:createrepo 命令用于創(chuàng)建yum源（軟件倉庫），即為存放于本地特定位置的眾多rpm包建立索引，描述各包所需依賴信息，并形成元數(shù)據(jù)。

yum install -y createrepo

在CM目錄下執(zhí)行 createrepo . 生成rpm源

[root@cdh01 CM]# createrepo . Spawning worker 0 with 1 pkgs Spawning worker 1 with 1 pkgs Spawning worker 2 with 1 pkgs Spawning worker 3 with 1 pkgs Spawning worker 4 with 1 pkgs Spawning worker 5 with 1 pkgs Spawning worker 6 with 0 pkgs Spawning worker 7 with 0 pkgs Spawning worker 8 with 0 pkgs Spawning worker 9 with 0 pkgs Spawning worker 10 with 0 pkgs Spawning worker 11 with 0 pkgs Spawning worker 12 with 0 pkgs Spawning worker 13 with 0 pkgs Spawning worker 14 with 0 pkgs Spawning worker 15 with 0 pkgs Spawning worker 16 with 0 pkgs Spawning worker 17 with 0 pkgs Spawning worker 18 with 0 pkgs Spawning worker 19 with 0 pkgs Spawning worker 20 with 0 pkgs Spawning worker 21 with 0 pkgs Spawning worker 22 with 0 pkgs Spawning worker 23 with 0 pkgs Workers Finished Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Sqlite DBs complete

安裝 httpd 服務

yum install -y httpd

啟動服務

systemctl start httpdsystemctl status httpd

將CM 與 CDH 目錄移動到目錄 /var/www/html/ 下

mv CM/ CDH/ /var/www/html/

驗證是否安裝成功，訪問 http://ip/CM ，http://ip/CDH

顯示如下即可

提示以下內容說明沒有成功，原因是?SELinux 未禁用成功，在所有服務器執(zhí)行?setenforce 0? 即可

配置安裝源

vim /etc/yum.repos.d/cm.repo [cmrepo] name = cm_repo baseurl = http://ip/CM enable = true gpgcheck = false

安裝JDK

yum install oracle-j2sdk1.8-1.8.0+update181-1.x86_64

安裝 Cloudera Manager Server

yum install -y cloudera-manager-server

到這里CM服務端安裝完成，先不要啟動，安裝數(shù)據(jù)庫

安裝MariaDB

? ? ? ? MariaDB數(shù)據(jù)庫管理系統(tǒng)是MySQL的一個分支，主要由開源社區(qū)在維護，采用GPL授權許可 MariaDB的目的是完全兼容MySQL，包括API和命令行，使之能輕松成為MySQL的代替品。在存儲引擎方面，使用XtraDB（英語：XtraDB）來代替MySQL的InnoDB。 MariaDB由MySQL的創(chuàng)始人Michael Widenius（英語：Michael Widenius）主導開發(fā)，他早前曾以10億美元的價格，將自己創(chuàng)建的公司MySQL AB賣給了SUN，此后，隨著SUN被甲骨文收購，MySQL的所有權也落入Oracle的手中。MariaDB名稱來自Michael Widenius的女兒Maria的名字。

? ? ? ?MariaDB基于事務的Maria存儲引擎，替換了MySQL的MyISAM存儲引擎，它使用了Percona的 XtraDB，InnoDB的變體，分支的開發(fā)者希望提供訪問即將到來的MySQL 5.4 InnoDB性能。這個版本還包括了 PrimeBase XT (PBXT) 和 FederatedX存儲引擎。

yum -y install mariadb yum -y install mariadb-server

啟動、配置MariaDB

systemctl start mariadb [root@cdh01 ~]# /usr/bin/mysql_secure_installationNOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDBSERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!In order to log into MariaDB to secure it, we'll need the current password for the root user. If you've just installed MariaDB, and you haven't set the root password yet, the password will be blank, so you should just press enter here.Enter current password for root (enter for none): OK, successfully used password, moving on...Setting the root password ensures that nobody can log into the MariaDB root user without the proper authorisation.Set root password? [Y/n] y #設置root密碼 New password: Re-enter new password: Password updated successfully! Reloading privilege tables..... Success!By default, a MariaDB installation has an anonymous user, allowing anyone to log into MariaDB without having to have a user account created for them. This is intended only for testing, and to make the installation go a bit smoother. You should remove them before moving into a production environment.Remove anonymous users? [Y/n] y #是否禁止匿名登錄... Success!Normally, root should only be allowed to connect from 'localhost'. This ensures that someone cannot guess at the root password from the network.Disallow root login remotely? [Y/n] n #是否禁止遠程登錄... skipping.By default, MariaDB comes with a database named 'test' that anyone can access. This is also intended only for testing, and should be removed before moving into a production environment.Remove test database and access to it? [Y/n] y #刪除測試數(shù)據(jù)庫- Dropping test database...... Success!- Removing privileges on test database...... Success!Reloading the privilege tables will ensure that all changes made so far will take effect immediately.Reload privilege tables now? [Y/n] y #加載最新設置... Success!Cleaning up...All done! If you've completed all of the above steps, your MariaDB installation should now be secure.Thanks for using MariaDB!

登錄mysql 創(chuàng)建數(shù)據(jù)庫

mysql -uroot -p123456 create database metastore default character set utf8; #創(chuàng)建數(shù)據(jù)庫名稱為metastore，默認編碼utf-8 CREATE USER 'hive'@'%' IDENTIFIED BY '123456'; #創(chuàng)建用戶名為hive的用戶，密碼為123456，可以在任何機器上訪問 GRANT ALL PRIVILEGES ON metastore. * TO 'hive'@'%'; #授權hive用戶metastore所有權限 FLUSH PRIVILEGES; #刷新配置create database cm default character set utf8; CREATE USER 'cm'@'%' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON cm. * TO 'cm'@'%'; FLUSH PRIVILEGES; create database hue default character set utf8; CREATE USER 'hue'@'%' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON hue. * TO 'hue'@'%'; FLUSH PRIVILEGES;create database oozie default character set utf8; CREATE USER 'oozie'@'%' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON oozie. * TO 'oozie'@'%'; FLUSH PRIVILEGES;create database sentry default character set utf8; CREATE USER 'sentry'@'%' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON sentry. * TO 'sentry'@'%'; FLUSH PRIVILEGES;

安裝Mysql驅動

此處不安裝，服務無法找到mysql驅動，會導致服務失敗，集群所有服務都安裝，避免以后遇到問題

[root@cdh01 cdh]# mkdir /usr/share/java [root@cdh01 cdh]# mv mysql-connector-java-5.1.30.jar /usr/share/java/ [root@cdh01 cdh]# chmod 777 /usr/share/java/mysql-connector-java-5.1.30.jar [root@cdh01 cdh]# ln -s /usr/share/java/mysql-connector-java-5.1.30.jar /usr/share/java/mysql-connector-java.jar [root@cdh01 cdh]# ll /usr/share/java/ 總用量 932 -rwxrwxrwx. 1 root root 954038 6月 19 11:28 mysql-connector-java-5.1.30.jar lrwxrwxrwx. 1 root root 47 9月 1 13:59 mysql-connector-java.jar -> /usr/share/java/mysql-connector-java-5.1.30.jar

Cloudera Manager Server 初始化數(shù)據(jù)庫

注意這里 123456 是我們上一步創(chuàng)建的數(shù)據(jù)庫密碼

[root@cdh01 ~]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql cm cm 123456 JAVA_HOME=/usr/local/java/jdk1.8.0_191 Verifying that we can write to /etc/cloudera-scm-server Creating SCM configuration file in /etc/cloudera-scm-server Executing: /usr/local/java/jdk1.8.0_191/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera/cm/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. [ main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly!

啟動Cloudera Manager Server 服務

systemctl start cloudera-scm-server systemctl status cloudera-scm-server

啟動可能需要半分鐘左右，可以查看日志啟動狀態(tài)，提示如下內容就啟動成功了

日志目錄：/var/log/cloudera-scm-server

2019-09-01 14:05:02,994 INFO WebServerImpl:org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@534fca49{HTTP/1.1,[http/1.1]}{0.0.0.0:7180} 2019-09-01 14:05:02,995 INFO WebServerImpl:org.eclipse.jetty.server.Server: Started @95217ms 2019-09-01 14:05:02,995 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

瀏覽器訪問，提示登錄，默認賬號密碼均為 admin

http://ip:7180/cmf/login

接受繼續(xù)

這里選擇免費版本(也可以選擇試用版，到期后自動恢復成免費版本)

開始安裝集群

?

選擇我們剛剛安裝的httpd服務鏈接

agent 安裝成功，下一步

安裝 parcel ，這個過程可能要等一段時間

?

?

這里我們看下檢查結果

有兩處異常，根據(jù)提示修改配置：

?sysctl vm.swappiness=10 修改CDH建議值，重新檢測。

解決了第一個問題

設置一下兩個配置

echo never > /sys/kernel/mm/transparent_hugepage/defrag echo never > /sys/kernel/mm/transparent_hugepage/enabled

再次查看，全部正常

?

根據(jù)需求自己分配角色

安裝成功

總結

以上是生活随笔為你收集整理的Centos7.x 安装 CDH 6.x的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內容還不錯，歡迎將生活随笔推薦給好友。