Windows下安装Cygwin配置Hadoop集群
Hadoop集群一般是配置到Linux系統(tǒng)之上，如果電腦用的是Windows系統(tǒng)，那么可以通過(guò)虛擬機(jī)安裝Linux系統(tǒng)或者在Windows上安裝Cygwin來(lái)模擬Linux環(huán)境，從而搭建Hadoop集群，用于開(kāi)發(fā)和測(cè)試也是可以的。
本文講的是在Windows下安裝Cygwin模擬Linux環(huán)境，從而配置Hadoop集群的方法。
一、下載Cygwin，點(diǎn)擊前往下載
二、在Windows上安裝Cygwin
安裝
1.雙擊setup.exe
2.按照安裝提示“下一步”
3.選擇從“互聯(lián)網(wǎng)”安裝：
4.選擇安裝的路徑，一般是C:\Cygwin（也可以安裝在別的路徑下），選擇本地臨時(shí)目錄，用于存放下載的軟件；選擇您連接互聯(lián)網(wǎng)的方式，一般選“直接連接”就可以了。
5.選擇下載源：（課選擇中國(guó)的鏡像，下載更快些）
6.根據(jù)安裝提示，選擇需要安裝的軟件包，完成安裝。
這里要勾選（帶叉號(hào)的方框表示被勾選，空的方框表示未被勾選）的軟件包有：
vim/vi
ssh(openssh) 【Net Category下】
ssl(openssl) ?【Net Category下】
apt-cyg
sed （若需要Eclipse，必須sed）【BaseCategory下】
subversion 【Devel Category下】，還有其他將來(lái)開(kāi)發(fā)需要用到的組件，展開(kāi)該組件，點(diǎn)擊前面的keep，使其變成版本號(hào)
Base組件的全部，點(diǎn)擊Base后面的Default，使其變?yōu)镮nstall
System組件下的util-linux組件，用于使用一些常用的more等功能進(jìn)行調(diào)試
點(diǎn)擊【下一步】繼續(xù)安裝。
雙擊桌面的Cygwin圖標(biāo)，即可啟動(dòng)Cygwin里的Bash。
或者，你可將C:/Cygwin/bin加到%PATH%當(dāng)中，您就可以直接在cmd.exe里面使用Linux命令了，比如less, cat, wc , wget
如果有一個(gè)Bash腳本，可以直接用C:/Cygwin/bin/bash.exe X:/scripts/Monitor.sh的形式調(diào)用。
三、Cygwin安裝完畢，下面配置Hadoop集群
1、在Windows上安裝Java JDK環(huán)境，并配置環(huán)境變量。（若Windows下已經(jīng)安裝了Java環(huán)境并配置好，則這一步略過(guò)）不用在Cygwin里安裝Java JDK，Cygwin用Windows上的Java JDK，并且不需要在Cygwin下的/etc/profile文件中設(shè)置Java路徑，在Hadoop/conf/hadoop-env.sh中也不需要設(shè)置Java環(huán)境。
2、可以在Cygwin中設(shè)置/etc/hosts和/etc/hostname，我這里沒(méi)有設(shè)置。
3、在Cygwin中配置SSH（重要！！）
_____________________
$ ssh-host-config
*** Query: Overwrite existing /etc/ssh_config file? (yes/no)yes
*** Info: Creating default /etc/ssh_config file
*** Query: Overwrite existing /etc/sshd_config file? (yes/no)yes
*** Info: Creating default /etc/sshd_config file
*** Info: Privilege separation is set to yes by default since OpenSSH 3.3.
*** Info: However, this requires a non-privileged account called 'sshd'.
*** Info: For more info on privilege separation read /usr/share/doc/openssh/README.privsep.
*** Query: Should privilege separation be used? (yes/no)yes
*** Info: Updating /etc/sshd_config file
*** Warning: The following functions require administrator privileges!
*** Query: Do you want to install sshd as a service?
*** Query: (Say"no"if it is already installed as a service) (yes/no)yes
*** Query: Enter the value of CYGWIN for the daemon: []ntsec tty
*** Info: On Windows Server 2003, Windows Vista, and above, the
*** Info: SYSTEM account cannot setuid to other users -- a capability
*** Info: sshd requires. You need to have or to create a privileged
*** Info: account. This script will help you do so.
*** Info: You appear to be running Windows XP 64bit, Windows 2003 Server,
*** Info: or later. On these systems, it's not possible to use the LocalSystem
*** Info: account for services that can change the user id without an
*** Info: explicit password (such as passwordless logins [e.g. public key
*** Info: authentication] via sshd).
*** Info: If you want to enable that functionality, it's required to create
*** Info: a new account with special privileges (unless a similar account
*** Info: already exists). This account is then used to run these special
*** Info: servers.
*** Info: Note that creating a new user requires that the current account
*** Info: No privileged account could be found.
*** Info: This script plans to use 'cyg_server'.
*** Info: 'cyg_server' will only be used by registered services.
*** Query: Do you want to use a different name? (yes/no)yes 【no也可以】
*** Query: Enter the new user name:dev
*** Query: Reenter:dev
*** Query: Create new privileged user account 'dev'? (yes/no)yes
*** Info: Please enter a password for new user dev. Please be sure
*** Info: that this password matches the password rules given on your system.
*** Info: Entering no password will exit the configuration.
*** Query: Please enter the password:
*** Query: Reenter:
*** Info: User 'dev' has been created with password 'passw0rd'.
*** Info: If you change the password, please remember also to change the
*** Info: password for the installed services which use (or will soon use)
*** Info: the 'dev' account.
*** Info: Also keep in mind that the user 'dev' needs read permissions
*** Info: on all users' relevant files for the services running as 'dev'.
*** Info: In particular, for the sshd server all users' .ssh/authorized_keys
*** Info: files must have appropriate permissions to allow public key
*** Info: authentication. (Re-)running ssh-user-config for each user will set
*** Info: these permissions corrently. [Similary restrictions apply, for
*** Info: instance, for .rhosts files if the rshd server is running, etc].
*** Info: The sshd service has been installed under the 'dev'
*** Info: account. To start the service now, call `net start sshd' or
*** Info: `cygrunsrv -S sshd'. Otherwise, it will start automatically
*** Info: after the next reboot.
*** Info: Host configuration finished. Have fun!
_____________________
4、Windows下-【運(yùn)行】-【services.msc】，找到【CYGWIN sshd】服務(wù)，并開(kāi)啟該服務(wù)，可以設(shè)置為自動(dòng)開(kāi)啟。
5、在Cygwin命令行中輸入ssh-keygen -t rsa生成密鑰，并復(fù)制到authorized_keys中，執(zhí)行相應(yīng)的一系列操作，具體參見(jiàn)Hadoop集群搭建。
6、在Cygwin下安裝并配置Hadoop文件，具體參見(jiàn)Hadoop集群搭建。
7、運(yùn)行WordCount實(shí)例，確認(rèn)CygWin下Hadoop集群配置成功，具體參見(jiàn)Hadoop集群搭建。
8、開(kāi)啟Hadoop進(jìn)程，在Windows中打開(kāi)瀏覽器中輸入localhost:50030和localhost:50070查看信息。
9、在Windows中配置Eclipse開(kāi)發(fā)環(huán)境，在Cygwin開(kāi)啟Hadoop的情況下，在Windows下用eclipse開(kāi)發(fā)MapReduce程序。參見(jiàn)Eclipse安裝Hadoop插件配置Hadoop開(kāi)發(fā)環(huán)境和Hadoop+eclipse運(yùn)行MapReduce程序。
參考：
[1]Cygwin的安裝
總結(jié)
以上是生活随笔為你收集整理的Windows下安装Cygwin配置Hadoop集群的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。
- 上一篇： 自己动手写一个简单的MVC框架（第二版）
- 下一篇： 开源日志系统比较