hadoop入门-在windows上编译x64位hadoop
環(huán)境
Windows 7 x64位，visual studio 2010 professional
hadoop 源碼版本2.2.0
步驟（來源于書《Pro Apache Hadoop, Second Edition》略作修改。
Download the hadoop-2.2.x-src.tar.gz files (2.2.0 at the time of this writing) from the download section of the Apache web site for Hadoop. The download link is as follows: http://www.apache.org/dist/hadoop/core/hadoop-2.2.0/hadoop-2.2.0-src.tar.gz.
Explode the tar file into a directory. For the purpose of this section, we assume that the directory is called c:/myapps/hadoop/.
If using Visual Studio, use Visual Studio 2010 Professional (not 2012). Do not use Visual Studio Express because it does not support compiling for 64-bit, which presents problems if running on a 64-bit system. Alternatively, download the Microsoft Windows SDK v7.1. At the time of writing, the link is the following:
http://www.microsoft.com/en-us/download/confirmation.aspx?id=8279.
Place the SDK in the folder with full privileges. For this section, we assume that the folder is c:/myapps/winsdk71/.
Download and install the Linux-like environment for Windows named Cygwin. We assume that Cygwin home folder is c:/myapps/cygwin/. Ensure that the following UNIX command-line tools are installed: sh, mkdir, rm, cp, tar, and gzip. The download location for Cygwin is http://cygwin.com/install.html.
Download and install Maven 3.1.1 or a compatible version. Installation consists of simply exploding the tar bundle into a Windows folder. We assume that the home folder for Maven download is c:/MyApps/apache-maven-3.1.1/. Verify that this folder has the bin subfolder.
Download and install Protocol Buffers 2.5.0. At the time of writing, the link is http://protobuf.googlecode.com/files/protoc-2.5.0-win32.zip.
Simply unzip the zip file into a folder (assumed to be c:/MyApps/protobuf/). Verify that the installation is correct by ensuring that the folder contains the protoc.exe file.
Install ZLIB from http://zlib.net/zlib128-dll.zip and explode the zip file in the c:/zlib-1.2.8 folder. At runtime, this folder must be accessible from the PATH variable.
Add the environment variables shown in Table A-1.
| JAVA_HOME | c:/MyApps/jdk16/ |
| M2_HOME | c:/MyApps/apache-maven-3.1.1/ |
| Platform | x64 (or Win32 when building on the 32-bit system) |
| CYGWIN_HOME | c:/myapps/cygwin/ |
| PROTOBUF_HOME | c:/myapps/protobuf/ |
| ZLIB_HOME | c:/zlib-1.2.8
12 .Add the following to the Windows PATH variable:
|
開始編譯，執(zhí)行下面的步驟：
hadoop 2.2.0在編譯的時候有一個bug，參考https://issues.apache.org/jira/browse/HADOOP-10110描述，然后下載https://issues.apache.org/jira/secure/attachment/12614482/HADOOP-10110.patch這個patch，按照這個patch修改hadoop-common-project/hadoop-auth/pom.xml文件，然后再進(jìn)行編譯，否則會出錯。
-
Change the folder to c:/MyApps/hadoop. Recall that this is the folder in which the Hadoop source was placed.
Execute the following command mvn package with the -Pdist,native-win -DskipTests -Dtar options. The exact command is this:
mvn package -Pdist,native-win -DskipTests –DtarIf there are no errors, the hadoop-2.2.0.tar.gz file is created in the c:/myapps/hadoop/ hadoop-dist/hadoop-2.2.0/ folder.
編譯完后，如果沒有任何錯誤，在hadoop-2.2.0-src\hadoop-dist\target目錄下面會生成hadoop-2.2.0.tar.gz文件。
第一步大功告成！
總結(jié)
以上是生活随笔為你收集整理的hadoop入门-在windows上编译x64位hadoop的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： BF,KMP,BM三种字符串匹配算法性能
- 下一篇： hadoop入门-在windows上安装