hadoop-0.20.2完全分布式集群

集群规划

准备五台台虚拟机（实验以五台RedHat Enterprise Linux 6.5为例）

防火墙、iptables、和SSH已经在上一篇说过在此就不再赘述，完全分布式相对于伪分布式多了几个注意点，我们按照以下规划步骤来做：

1. 配置hosts文件用于解析IP地址与主机名的映射
2. 建立hadoop运行账号
3. 配置SSH免密码登录
4. 下载解压hadoop的安装包
5. 配置jdk、hadoop环境变量
6. 配置hadoop-env.sh,mapred-site.xml,core-site.xml,hdfs-site.xml,masters,slaves.
7. 将配置好的文件复制到其他节点
8. 格式化namenode并启动hadoop集群
9. 检验集群健康情况

1.配置hosts文件

192.168.186.165 sishen.165
192.168.186.164 sishen.164
192.168.186.163 sishen.163
192.168.186.162 sishen.162
192.168.186.161 sishen.161

[root@sishen ~]# scp /etc/hosts sishen.162:/etc/

[root@sishen ~]# scp /etc/hosts sishen.163:/etc/

[root@sishen ~]# scp /etc/hosts sishen.164:/etc/

[root@sishen ~]# scp /etc/hosts sishen.165:/etc/

使用ping命令检验

[root@sishen ~]# ping -c 2 sishen.162
PING sishen.162 (192.168.186.162) 56(84) bytes of data.
64 bytes from sishen.162 (192.168.186.162): icmp_seq=1 ttl=64 time=29.2 ms
64 bytes from sishen.162 (192.168.186.162): icmp_seq=2 ttl=64 time=0.880 ms

--- sishen.162 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1012ms
rtt min/avg/max/mdev = 0.880/15.040/29.200/14.160 ms

2.建立hadoop运行账户

分别在五台机器上运行以下命令

[root@sishen ~]# useradd  hadoop
[root@sishen ~]# echo 123456 | passwd --stdin hadoop

建立hadoop的数据存放目录（注意：分别在五台机器上执行）

[root@sishen ~]# mkdir -p /hadoop-0.20.2/data
[root@sishen ~]# chown -R hadoop:hadoop /hadoop-0.20.2/
[root@sishen ~]# ll -d /hadoop-0.20.2/
drwxr-xr-x 3 hadoop hadoop 4096 Oct  9 14:33 /hadoop-0.20.2/

3.配置SSH免密码登录

分别在五台机器上运行以下命令

[root@sishen ~]# ssh-keygen

……

中间过程直接回车即可

…..

[root@sishen ~]# ssh-copy-id sishen.161

[root@sishen ~]# ssh-copy-id sishen.162

[root@sishen ~]# ssh-copy-id sishen.163

[root@sishen ~]# ssh-copy-id sishen.164

[root@sishen ~]# ssh-copy-id sishen.165

4.解压并配置hadoop软件包

解压hadoop压缩包

[root@sishen ~]# tar -xf hadoop-0.20.2.tar.gz -C /hadoop-0.20.2/

[root@sishen ~]# cd /hadoop-0.20.2/hadoop-0.20.2/conf/

开始配置hadoop
[root@sishen conf]# vim hadoop-env.sh

[root@sishen conf]# vim hdfs-site.xml

[root@sishen conf]# vim mapred-site.xml

[root@sishen conf]# vim core-site.xml

[root@sishen conf]# vim masters

[root@sishen conf]# vim slaves

[root@sishen hadoop-0.20.2]# scp hadoop.tar.gz sishen.162:/hadoop-0.20.2/

[root@sishen hadoop-0.20.2]# scp hadoop.tar.gz sishen.163:/hadoop-0.20.2/

[root@sishen hadoop-0.20.2]# scp hadoop.tar.gz sishen.164:/hadoop-0.20.2/

[root@sishen hadoop-0.20.2]# scp hadoop.tar.gz sishen.165:/hadoop-0.20.2/

启动hadoop集群

[root@sishen ~]# start-all.sh

检测守护进程

发现master上面如下进程

在salve节点上有如下进程

关闭集群

如有疏漏欢迎指正 ^_^!