CHD 5.10 离线安装
安裝虛擬機，我用的是centos6.5的版本。
選擇回車
重啟機器
設置虛擬機網絡
1.集群節點規劃,安裝三臺機器，每臺機器都執行以下操作，切記！
[root@hadoop-cdh1 ~]# vi /etc/sysconfig/networkNETWORKING=yes
HOSTNAME=hadoop-01? # 機器名稱
GATEWAY=192.168.137.2? # 網關
[root@hadoop-cdh1 ~]# vi /etc/hosts
127.0.0.1?? localhost localhost.localdomain localhost4 localhost4.localdomain4
::1???????? localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.137.130 hadoop-cdh1
192.168.137.131 hadoop-cdh2
192.168.137.132 hadoop-cdh3
[root@hadoop-cdh1 ~]# vi /etc/resolv.conf ?
search localdomain
nameserver 192.168.137.2
nameserver 8.8.8.8??????????????? #ping 外網
nameserver 8.8.4.4
[root@hadoop-cdh1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.137.130??? #ip
NETMASK=225.225.225.0
GATEWAY=192.168.137.2
/etc/udev/rules.d/70-persistent-net.rules ?
1) Linux操作系統中永久性生效，重啟后不會復原
開啟： chkconfig iptables on
關閉： chkconfig iptables off
2) 即時生效，重啟后復原
開啟： service iptables start
關閉： service iptables stop
service network restart 重啟網卡
service iptables status可以查看到iptables服務的當前狀態。
yum install -y lrzsz
yum -y install wget
yum -y install vim
三臺機器設置完成后，reboot重啟三臺機器，檢查機器配置。
hostname 是否正確。
hostname -i是否正確。
ping www .baidu.com是否可以ping通。
安裝mysql
https://blog.csdn.net/nszkadrgg/article/details/78666628
配置hadoop用戶的ssh信任關系(三臺機器都要安裝)
安裝無密碼登錄(三臺機器都要安裝)
[root@hadoop-cdh1 ~]# yum -y install openssh-clients
[root@hadoop-cdh1 java]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
02:97:cf:e5:85:f4:3d:77:4b:24:18:23:9d:47:0e:1b root@hadoop-cdh1
The key's randomart image is:
+--[ RSA 2048]----+
|???????? .oE=o . |
|?????? . ..**oo? |
|??? . o?? o.+.o.o|
|???? o o o .? .oo|
|????? . S .??? . |
|?????? .???????? |
|???????????????? |
|???????????????? |
|???????????????? |
+-----------------+
[root@hadoop-cdh1 java]# cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[root@hadoop-cdh1 java]# chmod 0600 ~/.ssh/authorized_keys
測試三臺機器ssh登錄(每臺都要測試)
[root@hadoop-cdh1 java]# ssh hadoop-cdh2
The authenticity of host 'hadoop-cdh2 (192.168.137.131)' can't be established.
RSA key fingerprint is 69:99:33:84:fe:91:f6:43:df:4b:0c:63:ba:9b:c1:fc.
Are you sure you want to continue connecting (yes/no)?
Host key verification failed.
[root@hadoop-cdh1 java]# ssh hadoop-cdh2
The authenticity of host 'hadoop-cdh2 (192.168.137.131)' can't be established.
RSA key fingerprint is 69:99:33:84:fe:91:f6:43:df:4b:0c:63:ba:9b:c1:fc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop-cdh2,192.168.137.131' (RSA) to the list of known hosts.
root@hadoop-cdh2's password:
Last login: Sat Apr 21 03:54:12 2018 from hadoop-cdh3
檢查系統版本是否定一致 (每臺都要做)
[root@hadoop-cdh2 ~]# cat /etc/redhat-release
CentOS release 6.5 (Final)
設置時鐘同步(每臺都要做)
[root@hadoop-cdh2 ~]# grep ZONE /etc/sysconfig/clock
ZONE="Asia/Shanghai"
修改機器的時間
[root@hadoop-cdh1 ~]# date
Sat Apr 21 04:07:25 CST 2018
[root@hadoop-cdh1 ~]# date -s 04/20/2018
Fri Apr 20 00:00:00 CST 2018
[root@hadoop-cdh1 ~]# date -s 20:32:00
Fri Apr 20 20:32:00 CST 2018
[root@hadoop-cdh1 ~]# dare
-bash: dare: command not found
[root@hadoop-cdh1 ~]# date
Fri Apr 20 20:32:06 CST 2018
安裝ntpdate(每臺機器都要安裝)
[root@hadoop-cdh3 ~]# yum -y install ntpdate
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
?* base: mirrors.zju.edu.cn
?* extras: mirrors.aliyun.com
?* updates: mirrors.njupt.edu.cn
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package ntpdate.x86_64 0:4.2.6p5-12.el6.centos.2 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=======================================================================================================================================================================================================================
?Package???????????????????????????????????????? Arch?????????????????????????????????????????? Version????????????????????????????????????????????????????????? Repository?????????????????????????????????????? Size
=======================================================================================================================================================================================================================
Installing:
?ntpdate???????????????????????????????????????? x86_64???????????????????????????????????????? 4.2.6p5-12.el6.centos.2????????????????????????????????????????? updates????????????????????????????????????????? 79 k
Transaction Summary
=======================================================================================================================================================================================================================
Install?????? 1 Package(s)
Total download size: 79 k
Installed size: 123 k
Downloading Packages:
ntpdate-4.2.6p5-12.el6.centos.2.x86_64.rpm????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? |? 79 kB???? 00:00??? ?
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
? Installing : ntpdate-4.2.6p5-12.el6.centos.2.x86_64????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? 1/1
? Verifying? : ntpdate-4.2.6p5-12.el6.centos.2.x86_64????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? 1/1
Installed:
? ntpdate.x86_64 0:4.2.6p5-12.el6.centos.2????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
Complete!
安裝ntp服務(三臺都要安裝)
yum -y install ntp
啟動ntpd服務(三臺機器都要啟動)
[root@hadoop-cdh1 ~]# service ntpd start
Starting ntpd:???????????????????????????????????????????? [? OK? ]
時間設置同步
[root@hadoop-cdh1 ~]# ntpdate hadoop-cdh2
20 Apr 20:41:06 ntpdate[2676]: no server suitable for synchronization found
[root@hadoop-cdh1 ~]# ntpdate hadoop-cdh3
20 Apr 20:41:06 ntpdate[2676]: no server suitable for synchronization found
安裝和啟動http服務
[root@hadoop-cdh1 ~]# rpm -qa|grep httpd
httpd-tools-2.2.15-60.el6.centos.6.x86_64
httpd-2.2.15-60.el6.centos.6.x86_64
[root@hadoop-cdh1 ~]#
其他兩臺沒有安裝，安裝hadoop-cdh2，hadoop-cdh3兩臺的http服務，安裝? yum install -y httpd
下面的三步三臺都做
[root@hadoop-cdh1 ~]# chkconfig --list|grep httpd
httpd???????? ??? ?0:off?? ?1:off?? ?2:on?? ?3:on?? ?4:on?? ?5:on?? ?6:off
[root@hadoop-cdh1 ~]# chkconfig httpd on
[root@hadoop-cdh1 ~]# chkconfig --list|grep httpd
httpd???????? ??? ?0:off?? ?1:off?? ?2:on?? ?3:on?? ?4:on?? ?5:on?? ?6:off
[root@hadoop-cdh1 ~]# service httpd start
Starting httpd:
安裝jdk，一定要用oracle版本的jdk，官網上可以下載，不要安裝linux自帶的openJDK。
[root@hadoop001 ~]# rz #上傳jdk-8u45-linux-x64.gz[root@hadoop-cdh1 java]# mkdir -p /usr/java
[root@hadoop-cdh1 java]# mv jdk-8u45-linux-x64.gz /usr/java
[root@hadoop-cdh1 java]# tar -xzvf jdk-8u45-linux-x64.gz
解壓后的jdk復制到hadoop-cdh2，hadoop-cdh3
其他的兩臺創建mkdir -p /usr/java目錄
scp到其他的兩臺機器
scp -r /usr/java/jdk1.8.0_45/ hadoop-cdh2:/usr/java
scp -r /usr/java/jdk1.8.0_45/ hadoop-cdh3:/usr/java
配置jdk的環境變量
[root@hadoop-cdh1 java]# vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_45
export PATH=$JAVA_HOME/bin:$PATH
[root@hadoop-cdh1 java]# source /etc/profile
[root@hadoop-cdh1 java]# java -version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
scp 到其他的兩臺機器
[root@hadoop-cdh1 java]# scp -r /etc/profile/ hadoop-cdh2:/etc/profile
[root@hadoop-cdh1 java]# scp -r /etc/profile/ hadoop-cdh3:/etc/profile
其他的兩臺機器source /ect/profile
接下來安裝CDH
http://archive.cloudera.com/cdh5/parcels/5.10.0/
到CDH官網下載對應版本的parcels文件
我們存在parcel的位置是 /var/www/html
[root@hadoop-cdh1 html]# mkdir parcels
[root@hadoop-cdh1 html]# cd parcels/
[root@hadoop-cdh1 parcels]# ll
total 1466572
-rw-r--r--. 1 root root 1501694035 Dec 10 19:14 CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel
-rw-r--r--. 1 root root???????? 41 Dec 10 18:51 CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha
-rw-r--r--. 1 root root????? 64807 Dec 10 19:15 manifest.json
[root@hadoop-cdh1 parcels]# pwd
/var/www/html/parcels
CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha1? 后面的1去掉，之前我去掉了
[root@hadoop-cdh1 parcels]# mv CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha1 CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha
校驗你下載的文件是不是全的，如果不全也是不會安裝成功的，所以一定要校驗。
[root@hadoop-cdh1 parcels]# sha1sum CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel
52f95da433f203a05c2fd33eb0f144e6a5c9d558? CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel
[root@hadoop-cdh1 parcels]# cat CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha
52f95da433f203a05c2fd33eb0f144e6a5c9d558
下載cm
http://archive.cloudera.com/cm5/repo-as-tarball/5.10.0/
校驗下載的文件
解壓
[root@hadoop-cdh1 html]# tar -zxvf cm5.10.0-centos6.tar.gz
然后創建和官網一樣的路徑
修改文件名稱
配置本地的yum源，集群自安裝的時候就會從本地down包，不會從官網down(三臺機器都要配置)
vim /etc/yum.repos.d/cloudrea-manager.repo?? 配置的內容一樣，沒有cloudrea-manager.repo文件就創建
在瀏覽器上查看是否配置成功，能打開說明配置成功。
官網參考鏈接
http://archive.cloudera.com/cdh5/parcels/5.10/
http://archive.cloudera.com/cm5/repo-as-tarball/5.10.0/
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10/
install server rpm in cm instance
install server
mysql創建元數據庫和用戶，兩個都創建
修改service和mysql的配置
vim /etc/cloudera-scm-server/db.properties? 修改后保存
添加mysql的驅動包
service 啟動的日志
用戶名密碼默認admin
下一步
下一步
下一步
下一步
下一步
下一步
下一步
下一步
這一步很重要，點擊下一步
開始分發到每臺機器
下一步
關閉大頁面
設置swap，三臺機器都要設置
重新Run Again
沒有黃色的提示了
現在安裝hadoop的三個核心組件 ，hdfs, yarn,zookeeper
下面設置機器的角色分配
角色分配:
192.168.137.130? hadoop-cdh1
192.168.137.131? hadoop-cdh2
192.168.137.132? hadoop-cdh3
第一階段角色:
hadoop-cdh1: mysql cm-server cm-agent
hadoop-cdh2: cm-agent
hadoop-cdh3: cm-agent
cm(5個服務):hadoop-cdh1
第二階段角色:
hdfs 、yarn 、zk
hadoop-cdh1: NN? DN??? RM NM? ZK
hadoop-cdh2: DN? SNN?? NM???? ZK
hadoop-cdh3: DN??????? NM???? ZK
選擇下一步，這里一定要有mysql 的驅動包，目錄就按照下面的目錄放置，別的目錄有時候會找不到
[root@hadoop-cdh1 cloudera-manager]# cd /usr/share/java/
[root@hadoop-cdh1 java]# ll
total 964
-rw-r--r--. 1 root root 985603 Oct 22? 2017 mysql-connector-java.jar
[root@hadoop-cdh1 java]#
點擊下一步，所有的都不要變
點擊下一步，安裝服務
安裝完成，點擊完成。
設置namenode最小1G。
查找heap，點擊菜單的namenode，設置為1G，點擊保存。
返回主菜單
重啟
下一步
點擊Finish
點擊Finish
其他的黃色和紅色的信息，根據你自己集群的情況，如果資源充足就設置，不充足就抑制。
service cloudera-scm-agent status 查看agent的狀態(每臺)
service cloudera-scm-service status 查看server的狀態
正確的關機順序
service cloudera-scm-agent stop? 先關閉agent
service cloudera-scm-service stop? 再關閉service
service mysql? stop? 再關閉mysql
開機順序剛好相反，先啟動mysql，再啟動server，然后再啟動agent。
總結
以上是生活随笔為你收集整理的CHD 5.10 离线安装的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： docker的/var/run/dock
- 下一篇： 通过代理上网，如何配置Outlook