CHD 5.10 离线安装
安裝虛擬機(jī),我用的是centos6.5的版本。
選擇回車
重啟機(jī)器
設(shè)置虛擬機(jī)網(wǎng)絡(luò)
1.集群節(jié)點(diǎn)規(guī)劃,安裝三臺(tái)機(jī)器,每臺(tái)機(jī)器都執(zhí)行以下操作,切記!
[root@hadoop-cdh1 ~]# vi /etc/sysconfig/networkNETWORKING=yes
HOSTNAME=hadoop-01? # 機(jī)器名稱
GATEWAY=192.168.137.2? # 網(wǎng)關(guān)
[root@hadoop-cdh1 ~]# vi /etc/hosts
127.0.0.1?? localhost localhost.localdomain localhost4 localhost4.localdomain4
::1???????? localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.137.130 hadoop-cdh1
192.168.137.131 hadoop-cdh2
192.168.137.132 hadoop-cdh3
[root@hadoop-cdh1 ~]# vi /etc/resolv.conf ?
search localdomain
nameserver 192.168.137.2
nameserver 8.8.8.8??????????????? #ping 外網(wǎng)
nameserver 8.8.4.4
[root@hadoop-cdh1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.137.130??? #ip
NETMASK=225.225.225.0
GATEWAY=192.168.137.2
/etc/udev/rules.d/70-persistent-net.rules ?
1) Linux操作系統(tǒng)中永久性生效,重啟后不會(huì)復(fù)原
開啟: chkconfig iptables on
關(guān)閉: chkconfig iptables off
2) 即時(shí)生效,重啟后復(fù)原
開啟: service iptables start
關(guān)閉: service iptables stop
service network restart 重啟網(wǎng)卡
service iptables status可以查看到iptables服務(wù)的當(dāng)前狀態(tài)。
yum install -y lrzsz
yum -y install wget
yum -y install vim
三臺(tái)機(jī)器設(shè)置完成后,reboot重啟三臺(tái)機(jī)器,檢查機(jī)器配置。
hostname 是否正確。
hostname -i是否正確。
ping www .baidu.com是否可以ping通。
安裝mysql
https://blog.csdn.net/nszkadrgg/article/details/78666628
配置hadoop用戶的ssh信任關(guān)系(三臺(tái)機(jī)器都要安裝)
安裝無密碼登錄(三臺(tái)機(jī)器都要安裝)
[root@hadoop-cdh1 ~]# yum -y install openssh-clients
[root@hadoop-cdh1 java]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
02:97:cf:e5:85:f4:3d:77:4b:24:18:23:9d:47:0e:1b root@hadoop-cdh1
The key's randomart image is:
+--[ RSA 2048]----+
|???????? .oE=o . |
|?????? . ..**oo? |
|??? . o?? o.+.o.o|
|???? o o o .? .oo|
|????? . S .??? . |
|?????? .???????? |
|???????????????? |
|???????????????? |
|???????????????? |
+-----------------+
[root@hadoop-cdh1 java]# cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[root@hadoop-cdh1 java]# chmod 0600 ~/.ssh/authorized_keys
測(cè)試三臺(tái)機(jī)器ssh登錄(每臺(tái)都要測(cè)試)
[root@hadoop-cdh1 java]# ssh hadoop-cdh2
The authenticity of host 'hadoop-cdh2 (192.168.137.131)' can't be established.
RSA key fingerprint is 69:99:33:84:fe:91:f6:43:df:4b:0c:63:ba:9b:c1:fc.
Are you sure you want to continue connecting (yes/no)?
Host key verification failed.
[root@hadoop-cdh1 java]# ssh hadoop-cdh2
The authenticity of host 'hadoop-cdh2 (192.168.137.131)' can't be established.
RSA key fingerprint is 69:99:33:84:fe:91:f6:43:df:4b:0c:63:ba:9b:c1:fc.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop-cdh2,192.168.137.131' (RSA) to the list of known hosts.
root@hadoop-cdh2's password:
Last login: Sat Apr 21 03:54:12 2018 from hadoop-cdh3
檢查系統(tǒng)版本是否定一致 (每臺(tái)都要做)
[root@hadoop-cdh2 ~]# cat /etc/redhat-release
CentOS release 6.5 (Final)
設(shè)置時(shí)鐘同步(每臺(tái)都要做)
[root@hadoop-cdh2 ~]# grep ZONE /etc/sysconfig/clock
ZONE="Asia/Shanghai"
修改機(jī)器的時(shí)間
[root@hadoop-cdh1 ~]# date
Sat Apr 21 04:07:25 CST 2018
[root@hadoop-cdh1 ~]# date -s 04/20/2018
Fri Apr 20 00:00:00 CST 2018
[root@hadoop-cdh1 ~]# date -s 20:32:00
Fri Apr 20 20:32:00 CST 2018
[root@hadoop-cdh1 ~]# dare
-bash: dare: command not found
[root@hadoop-cdh1 ~]# date
Fri Apr 20 20:32:06 CST 2018
安裝ntpdate(每臺(tái)機(jī)器都要安裝)
[root@hadoop-cdh3 ~]# yum -y install ntpdate
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
?* base: mirrors.zju.edu.cn
?* extras: mirrors.aliyun.com
?* updates: mirrors.njupt.edu.cn
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package ntpdate.x86_64 0:4.2.6p5-12.el6.centos.2 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=======================================================================================================================================================================================================================
?Package???????????????????????????????????????? Arch?????????????????????????????????????????? Version????????????????????????????????????????????????????????? Repository?????????????????????????????????????? Size
=======================================================================================================================================================================================================================
Installing:
?ntpdate???????????????????????????????????????? x86_64???????????????????????????????????????? 4.2.6p5-12.el6.centos.2????????????????????????????????????????? updates????????????????????????????????????????? 79 k
Transaction Summary
=======================================================================================================================================================================================================================
Install?????? 1 Package(s)
Total download size: 79 k
Installed size: 123 k
Downloading Packages:
ntpdate-4.2.6p5-12.el6.centos.2.x86_64.rpm????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? |? 79 kB???? 00:00??? ?
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
? Installing : ntpdate-4.2.6p5-12.el6.centos.2.x86_64????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? 1/1
? Verifying? : ntpdate-4.2.6p5-12.el6.centos.2.x86_64????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? 1/1
Installed:
? ntpdate.x86_64 0:4.2.6p5-12.el6.centos.2????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????
Complete!
安裝ntp服務(wù)(三臺(tái)都要安裝)
yum -y install ntp
啟動(dòng)ntpd服務(wù)(三臺(tái)機(jī)器都要啟動(dòng))
[root@hadoop-cdh1 ~]# service ntpd start
Starting ntpd:???????????????????????????????????????????? [? OK? ]
時(shí)間設(shè)置同步
[root@hadoop-cdh1 ~]# ntpdate hadoop-cdh2
20 Apr 20:41:06 ntpdate[2676]: no server suitable for synchronization found
[root@hadoop-cdh1 ~]# ntpdate hadoop-cdh3
20 Apr 20:41:06 ntpdate[2676]: no server suitable for synchronization found
安裝和啟動(dòng)http服務(wù)
[root@hadoop-cdh1 ~]# rpm -qa|grep httpd
httpd-tools-2.2.15-60.el6.centos.6.x86_64
httpd-2.2.15-60.el6.centos.6.x86_64
[root@hadoop-cdh1 ~]#
其他兩臺(tái)沒有安裝,安裝hadoop-cdh2,hadoop-cdh3兩臺(tái)的http服務(wù),安裝? yum install -y httpd
下面的三步三臺(tái)都做
[root@hadoop-cdh1 ~]# chkconfig --list|grep httpd
httpd???????? ??? ?0:off?? ?1:off?? ?2:on?? ?3:on?? ?4:on?? ?5:on?? ?6:off
[root@hadoop-cdh1 ~]# chkconfig httpd on
[root@hadoop-cdh1 ~]# chkconfig --list|grep httpd
httpd???????? ??? ?0:off?? ?1:off?? ?2:on?? ?3:on?? ?4:on?? ?5:on?? ?6:off
[root@hadoop-cdh1 ~]# service httpd start
Starting httpd:
安裝jdk,一定要用oracle版本的jdk,官網(wǎng)上可以下載,不要安裝linux自帶的openJDK。
[root@hadoop001 ~]# rz #上傳jdk-8u45-linux-x64.gz[root@hadoop-cdh1 java]# mkdir -p /usr/java
[root@hadoop-cdh1 java]# mv jdk-8u45-linux-x64.gz /usr/java
[root@hadoop-cdh1 java]# tar -xzvf jdk-8u45-linux-x64.gz
解壓后的jdk復(fù)制到hadoop-cdh2,hadoop-cdh3
其他的兩臺(tái)創(chuàng)建mkdir -p /usr/java目錄
scp到其他的兩臺(tái)機(jī)器
scp -r /usr/java/jdk1.8.0_45/ hadoop-cdh2:/usr/java
scp -r /usr/java/jdk1.8.0_45/ hadoop-cdh3:/usr/java
配置jdk的環(huán)境變量
[root@hadoop-cdh1 java]# vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_45
export PATH=$JAVA_HOME/bin:$PATH
[root@hadoop-cdh1 java]# source /etc/profile
[root@hadoop-cdh1 java]# java -version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
scp 到其他的兩臺(tái)機(jī)器
[root@hadoop-cdh1 java]# scp -r /etc/profile/ hadoop-cdh2:/etc/profile
[root@hadoop-cdh1 java]# scp -r /etc/profile/ hadoop-cdh3:/etc/profile
其他的兩臺(tái)機(jī)器source /ect/profile
接下來安裝CDH
http://archive.cloudera.com/cdh5/parcels/5.10.0/
到CDH官網(wǎng)下載對(duì)應(yīng)版本的parcels文件
我們存在parcel的位置是 /var/www/html
[root@hadoop-cdh1 html]# mkdir parcels
[root@hadoop-cdh1 html]# cd parcels/
[root@hadoop-cdh1 parcels]# ll
total 1466572
-rw-r--r--. 1 root root 1501694035 Dec 10 19:14 CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel
-rw-r--r--. 1 root root???????? 41 Dec 10 18:51 CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha
-rw-r--r--. 1 root root????? 64807 Dec 10 19:15 manifest.json
[root@hadoop-cdh1 parcels]# pwd
/var/www/html/parcels
CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha1? 后面的1去掉,之前我去掉了
[root@hadoop-cdh1 parcels]# mv CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha1 CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha
校驗(yàn)?zāi)阆螺d的文件是不是全的,如果不全也是不會(huì)安裝成功的,所以一定要校驗(yàn)。
[root@hadoop-cdh1 parcels]# sha1sum CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel
52f95da433f203a05c2fd33eb0f144e6a5c9d558? CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel
[root@hadoop-cdh1 parcels]# cat CDH-5.10.0-1.cdh5.10.0.p0.41-el6.parcel.sha
52f95da433f203a05c2fd33eb0f144e6a5c9d558
下載cm
http://archive.cloudera.com/cm5/repo-as-tarball/5.10.0/
校驗(yàn)下載的文件
解壓
[root@hadoop-cdh1 html]# tar -zxvf cm5.10.0-centos6.tar.gz
然后創(chuàng)建和官網(wǎng)一樣的路徑
修改文件名稱
配置本地的yum源,集群自安裝的時(shí)候就會(huì)從本地down包,不會(huì)從官網(wǎng)down(三臺(tái)機(jī)器都要配置)
vim /etc/yum.repos.d/cloudrea-manager.repo?? 配置的內(nèi)容一樣,沒有cloudrea-manager.repo文件就創(chuàng)建
在瀏覽器上查看是否配置成功,能打開說明配置成功。
官網(wǎng)參考鏈接
http://archive.cloudera.com/cdh5/parcels/5.10/
http://archive.cloudera.com/cm5/repo-as-tarball/5.10.0/
http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.10/
install server rpm in cm instance
install server
mysql創(chuàng)建元數(shù)據(jù)庫和用戶,兩個(gè)都創(chuàng)建
修改service和mysql的配置
vim /etc/cloudera-scm-server/db.properties? 修改后保存
添加mysql的驅(qū)動(dòng)包
service 啟動(dòng)的日志
用戶名密碼默認(rèn)admin
下一步
下一步
下一步
下一步
下一步
下一步
下一步
下一步
這一步很重要,點(diǎn)擊下一步
開始分發(fā)到每臺(tái)機(jī)器
下一步
關(guān)閉大頁面
設(shè)置swap,三臺(tái)機(jī)器都要設(shè)置
重新Run Again
沒有黃色的提示了
現(xiàn)在安裝hadoop的三個(gè)核心組件 ,hdfs, yarn,zookeeper
下面設(shè)置機(jī)器的角色分配
角色分配:
192.168.137.130? hadoop-cdh1
192.168.137.131? hadoop-cdh2
192.168.137.132? hadoop-cdh3
第一階段角色:
hadoop-cdh1: mysql cm-server cm-agent
hadoop-cdh2: cm-agent
hadoop-cdh3: cm-agent
cm(5個(gè)服務(wù)):hadoop-cdh1
第二階段角色:
hdfs 、yarn 、zk
hadoop-cdh1: NN? DN??? RM NM? ZK
hadoop-cdh2: DN? SNN?? NM???? ZK
hadoop-cdh3: DN??????? NM???? ZK
選擇下一步,這里一定要有mysql 的驅(qū)動(dòng)包,目錄就按照下面的目錄放置,別的目錄有時(shí)候會(huì)找不到
[root@hadoop-cdh1 cloudera-manager]# cd /usr/share/java/
[root@hadoop-cdh1 java]# ll
total 964
-rw-r--r--. 1 root root 985603 Oct 22? 2017 mysql-connector-java.jar
[root@hadoop-cdh1 java]#
點(diǎn)擊下一步,所有的都不要變
點(diǎn)擊下一步,安裝服務(wù)
安裝完成,點(diǎn)擊完成。
設(shè)置namenode最小1G。
查找heap,點(diǎn)擊菜單的namenode,設(shè)置為1G,點(diǎn)擊保存。
返回主菜單
重啟
下一步
點(diǎn)擊Finish
點(diǎn)擊Finish
其他的黃色和紅色的信息,根據(jù)你自己集群的情況,如果資源充足就設(shè)置,不充足就抑制。
service cloudera-scm-agent status 查看agent的狀態(tài)(每臺(tái))
service cloudera-scm-service status 查看server的狀態(tài)
正確的關(guān)機(jī)順序
service cloudera-scm-agent stop? 先關(guān)閉agent
service cloudera-scm-service stop? 再關(guān)閉service
service mysql? stop? 再關(guān)閉mysql
開機(jī)順序剛好相反,先啟動(dòng)mysql,再啟動(dòng)server,然后再啟動(dòng)agent。
總結(jié)
以上是生活随笔為你收集整理的CHD 5.10 离线安装的全部?jī)?nèi)容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: docker的/var/run/dock
- 下一篇: 通过代理上网,如何配置Outlook