當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

hadoop-2.5.1 源码编译

發布時間：2023/12/15 编程问答 26 豆豆

生活随笔收集整理的這篇文章主要介紹了 hadoop-2.5.1 源码编译小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

hadoop-2.5.1編譯步驟(只需一個節點即可)

環境：	系統?? CentOS 6.4 64 位?? m1????? 172.103.55.12 軟件：? jdk1.7、hadoop- 2.5.1- src.tar.gz、maven3.2.3、protobuf2.5.0

注：需要連接互聯網
?

1.???????安裝配置JDK（略）

2.???????安裝基本應用程序?(需要連接互聯網網)?

yum -y install??svn? ?ncurses-devel? ?gcc*? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?
yum -y install lzo-devel zlib-devel autoconf? ? automake? ? libtool? ? cmake? ???openssl –devel

3.??????安裝?maven

下載 apache-maven-3.2.3-bin.tar.gz? ?
并解壓 tar? ?zxvf? ?apache- maven- 3.2.3- bin.tar.gz

配置環境變量
export MAVEN_HOME=/usr/local/program/maven/ apache- maven- 3.2.3
export PATH=$PATH:$MAVEN_HOME/bin? ?? ??
添加之后執行sourc?/etc/profile

4.?????安裝?protobuf（不安裝，編譯將無法完成）

hadoop使用protocol buffer進行通信，需要下載和安裝protobhf-2.5.0.tar.gz；由于官網已經無法下載了，

可以到百度云盤下載http://pan.baidu.com/s/1pJlZubT

編譯安裝?protobuf
①??cd? ?protobuf - 2.5.0? ?
②??./configure?
③??make
④??make install

檢測??protoc –version

5.?????編譯?hadoop-2.5.1.src

a)?????下載??hadoop-2.5.1-src.tat.gz
?源碼包下載地址??http://apache.claz.org/hadoop/common/hadoop-2.5.1/
?紅色標志??hadoop-2.5.1- src.tar.gz?為源碼包
?

b)????解壓hadoop-2.5.1-src.tar.gz

c)?????開始編譯

進入hadoop-2.5.1.src 目錄：

執行命令? ?mvn clean package –Pdist,native –DskipTests –Dtar?

（或者 mvn?package?-Pdist,native?-DskipTests -Dtar）?

????
此步驟要求逐級能訪問公網，并且此步驟比較漫長，請耐心等待。

經過漫長的等待后，終于看到了Building SUCCESS:

[INFO] Reactor Summary:
[INFO]?
[INFO] Apache Hadoop Main ................................. SUCCESS [ ?4.389 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ ?3.163 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ ?6.377 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ ?0.562 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ ?3.136 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ ?6.757 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ ?5.506 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [ ?6.269 s]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ ?4.341 s]
[INFO] Apache Hadoop Common ............................... SUCCESS [03:25 min]
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 23.998 s]
[INFO] Apache Hadoop Common Project ....................... SUCCESS [ ?0.160 s]
[INFO] Apache Hadoop HDFS ................................. SUCCESS [07:52 min]
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [11:03 min]
[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [03:23 min]
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 14.324 s]
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [ ?0.343 s]
[INFO] hadoop-yarn ........................................ SUCCESS [ ?0.092 s]
[INFO] hadoop-yarn-api .................................... SUCCESS [02:46 min]
[INFO] hadoop-yarn-common ................................. SUCCESS [02:37 min]
[INFO] hadoop-yarn-server ................................. SUCCESS [ ?1.142 s]
[INFO] hadoop-yarn-server-common .......................... SUCCESS [01:02 min]
[INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [02:26 min]
[INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [ 22.258 s]
[INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [ 16.444 s]
[INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 51.768 s]
[INFO] hadoop-yarn-server-tests ........................... SUCCESS [ ?9.864 s]
[INFO] hadoop-yarn-client ................................. SUCCESS [ 16.057 s]
[INFO] hadoop-yarn-applications ........................... SUCCESS [ ?0.162 s]
[INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [ ?7.307 s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [ ?3.243 s]
[INFO] hadoop-yarn-site ................................... SUCCESS [ ?0.099 s]
[INFO] hadoop-yarn-project ................................ SUCCESS [ 18.997 s]
[INFO] hadoop-mapreduce-client ............................ SUCCESS [ ?2.306 s]
[INFO] hadoop-mapreduce-client-core ....................... SUCCESS [01:34 min]
[INFO] hadoop-mapreduce-client-common ..................... SUCCESS [01:05 min]
[INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [ 20.109 s]
[INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 22.691 s]
[INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 30.368 s]
[INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [01:30 min]
[INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [ 19.958 s]
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 24.097 s]
[INFO] hadoop-mapreduce ................................... SUCCESS [ 17.757 s]
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [01:59 min]
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 48.464 s]
[INFO] Apache Hadoop Archives ............................. SUCCESS [ 14.187 s]
[INFO] Apache Hadoop Rumen ................................ SUCCESS [ 15.098 s]
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 13.630 s]
[INFO] Apache Hadoop Data Join ............................ SUCCESS [ ?8.080 s]
[INFO] Apache Hadoop Extras ............................... SUCCESS [ ?6.496 s]
[INFO] Apache Hadoop Pipes ................................ SUCCESS [ 23.855 s]
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 17.886 s]
[INFO] Apache Hadoop Client ............................... SUCCESS [ 18.728 s]
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ ?1.060 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 35.296 s]
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 13.476 s]
[INFO] Apache Hadoop Tools ................................ SUCCESS [ ?0.134 s]
[INFO] Apache Hadoop Distribution ......................... SUCCESS [01:20 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 51:55 min
[INFO] Finished at: 2014-12-09T19:16:49-08:00
[INFO] Final Memory: 142M/350M
[INFO] ------------------------------------------------------------------------

編譯成功后，??/hadoop-dist/target/hadoop-2.5.1.tar.gz? ?就是我們需要的文件了；

?可能遇到的問題：

報錯[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) on project hadoop-hdfs: Fatal error compiling: Error while executing the compiler. InvocationTargetException: Java heap space

分配內存不足,參考如下為maven配置JVM參數:?export MAVEN_OPTS=”-Xms256m -Xmx512m -Djava.awt.headless=true”

我直接在profile文件中：export MAVEN_OPTS=-Xmx1024

創作挑戰賽新人創作獎勵來咯，堅持創作打卡瓜分現金大獎

總結

以上是生活随笔為你收集整理的hadoop-2.5.1 源码编译的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

上一篇： bing工具栏一直自动更新怎么处理 bi
下一篇： MySql允许被远程访问