單機部署

官網文檔

https://hadoop.apache.org/docs/r2.9.2/hadoop-project-dist/hadoop-common/SingleCluster.html

單節點配置

Configure parameters as follows:
etc/hadoop/mapred-site.xml:

<configuration><property><name>mapreduce.framework.name</name><value>yarn</value></property> </configuration>

etc/hadoop/yarn-site.xml:

<configuration><property><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle</value></property> </configuration>

Start ResourceManager daemon and NodeManager daemon:

$ sbin/start-yarn.sh
Browse the web interface for the ResourceManager; by default it is available at:

ResourceManager - http://localhost:8088/
Run a MapReduce job.

When you’re done, stop the daemons with:

$ sbin/stop-yarn.sh

HA分布式集群部署

官網文檔

https://hadoop.apache.org/docs/r2.9.2/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html

配置

etc/hadoop/yarn-site.xml:

<property><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle</value> </property> <property><name>yarn.resourcemanager.ha.enabled</name><value>true</value> </property> <property><name>yarn.resourcemanager.cluster-id</name><value>cluster1</value> </property> <property><name>yarn.resourcemanager.ha.rm-ids</name><value>rm1,rm2</value> </property> <property><name>yarn.resourcemanager.hostname.rm1</name><value>master1</value> </property> <property><name>yarn.resourcemanager.hostname.rm2</name><value>master2</value> </property> <property><name>yarn.resourcemanager.webapp.address.rm1</name><value>master1:8088</value> </property> <property><name>yarn.resourcemanager.webapp.address.rm2</name><value>master2:8088</value> </property> <property><name>yarn.resourcemanager.zk-address</name><value>zk1:2181,zk2:2181,zk3:2181</value> </property>

啟動yarn集群

訪問yarn

總結

以上是生活随笔為你收集整理的hadoop yarn集群HA安装及配置的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。