hadoop-2.2.0 的编译安装及HA配置

一准备工作

准备工作中要求有

1.centOs 6.4，添加hadoop用户，配置集群内的/etc/hosts文件。

2.安装hadoop用户的ssh，并打通集群内所有机器，（ha执行fencing时会用到）。

3.下载社区版hadoop-2.2.0源码。

二编译hadoop 2.2.0

（编译hadoop 2.2.0所需要的软件可在此处下载：http://pan.baidu.com/s/1mgodf40）

--------------------------------------------------------------------------------------------
yum -y install  lzo-devel  zlib-devel  gcc autoconf automake libtool gcc-c++
yum install openssl-devel
yum install ncurses-devel
--------------------------------------------------------------------------------------------
Ant Maven ProtocolBuffer
findbugs   CMake

#安装java

yum -y install jdk

Protobuf 编译安装
tar -zxvf protobuf-2.5.0.tar.gz cd protobuf-2.5.0
./configure --prefix=/usr/local/protobuf make make install
Ant 安装
tar -zxvf apache-ant-1.9.2-bin.tar.gz mv apache-ant-1.9.2/ /usr/local/ant
maven 安装
tar -zxvf apache-maven-3.0.5-bin.tar.gz mv apache-maven-3.0.5/ /usr/local/maven
findbugs 安装
tar -zxfv findbugs-2.0.2.tar.gz
mv findbugs-2.0.2/ /usr/local/findbugs
cmake 编译安装
tar -zvxf cmake-2.8.6.tar.gz cd cmake-2.8.6 ./bootstrap make make install

-------------------------------------------------------------------------------------------- 配置环境

#根据自己的环境具体配置
vi /etc/profile  #java
export JAVA_HOME=/usr/java/jdk1.7.0_45 export JRE_HOME=/usr/java/jdk1.7.0_45/jre
export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib:$JRE_HOME/lib export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin  #maven
export MAVEN_HOME=/usr/local/maven export MAVEN_OPTS="-Xms256m -Xmx512m" export CLASSPATH=.:$CLASSPATH:$MAVEN_HOME/lib export PATH=$PATH:$MAVEN_HOME/bin
#protobuf
export PROTOBUF_HOME=/usr/local/protobuf
export CLASSPATH=.:$CLASSPATH:$PROTOBUF_HOME/lib export PATH=$PATH:$PROTOBUF_HOME/bin   #ant
export ANT_HOME=/usr/local/ant
export CLASSPATH=.:$CLASSPATH:$ANT_HOME/lib export PATH=$PATH:$ANT_HOME/bin
#findbugs
export FINDBUGS_HOME=/usr/local/findbugs
export CLASSPATH=.:$CLASSPATH:$FINDBUGS_HOME/lib export PATH=$PATH:$FINDBUGS_HOME/bin
source /etc/profile
--------------------------------------------------------------------------------------------
vi /hadoop-2.2.0/hadoop-common-project/hadoop-auth/pom.xml
<dependency>
<groupid>org.mortbay.jetty</groupid>

<artifactid>jetty</artifactid> <scope>test</scope> </dependency>
在上面代码后添加下面代码 <dependency>
<groupid>org.mortbay.jetty</groupid> <artifactid>jetty-util</artifactid> <scope>test</scope> </dependency>
注：不更改可能报下面错位 [ERROR]
Failed
to
execute
goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:
----------------------------------------------------------------------------------------------
重新编译：
tar -zvxf hadoop-2.2.0-src.tar cd hadoop-2.2.0-src
mvn clean package -DskipTests -Pdist,native,docs -Dtar  # 漫长等待

（注：可能存在glibc版本问题，此类问题网上有较多评论可供参考）

三安装hadoop

解压 hadoop-2.2.0.tar.gz

配置 hadoop用户的 ~/.bashrc 如下：

# User specific environment and startup programs
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_CONF_DIR=/usr/local/common/hadoop/conf
export HBASE_HOME=/usr/local/hbase
export HBASE_CONF_DIR=/usr/local/common/hbase/conf

export JAVA_HOME=/usr/java
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib/rt.jar
export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HBASE_HOME/bin:/usr/local/zookeeper/bin:/data1/script

alias jps="jps -J-Djava.io.tmpdir=$HOME"
alias jstat="jstat -J-Djava.io.tmpdir=$HOME"

source ~/.bashrc

四配置hadoop

在 $HADOOP_CONF_DIR目录下编辑hadoop的配置文件。

#配置 hdfs-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>
<property>
<name>dfs.nameservices</name>
<value>hbaseCluster</value>
</property>
<property>
<name>dfs.ha.namenodes.hbaseCluster</name>
<value>nn1,nn2</value>
</property>

<property>
<name>dfs.namenode.rpc-address.hbaseCluster.nn1</name>
<value>h112191.mars.grid.sina.com.cn:8020</value>
</property>
<property>
<name>dfs.namenode.http-address.hbaseCluster.nn1</name>
<value>h112191.mars.grid.sina.com.cn:50070</value>
</property>
<property>
<name>dfs.namenode.rpc-address.hbaseCluster.nn2</name>
<value>h112192.mars.grid.sina.com.cn:8020</value>
</property>
<property>
<name>dfs.namenode.http-address.hbaseCluster.nn2</name>
<value>h112192.mars.grid.sina.com.cn:50070</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>file:///data1/hadoop/namenode_nfs</value>
<description>指定用于HA存放edits的共享存储，通常是NFS挂载点</description>
</property>

<property>
<name>dfs.replication</name>
<value>3</value>
<final>true</final>
</property>
<property>
<name>dfs.name.dir</name>
<value>file:///data1/hadoop/namenode</value>
<final>true</final>
</property>
<property>
<name>dfs.data.dir</name>
<value>/data11/hadoop/data/datanode,/data2/hadoop/data/datanode,/data3/hadoop/data/datanode,/data4/hadoop/data/datanode,/data5/hadoop/data/datanode,/data6/hadoop/data/datanode,/data7/hadoop/data/datanode,/data8/hadoop/data/datanode,/data9/hadoop/data/datanode,/data10/hadoop/data/datanode</value>
<final>true</final>
</property>
<property>
<name>fs.checkpoint.dir</name>
<value>/data1/hadoop/namesecondary</value>
<final>true</final>
</property>
<property>
<name>dfs.block.size</name>
<value>134217728</value>
<final>true</final>
</property>

<property>
<name>dfs.hosts</name>
<value>/usr/local/common/hadoop/conf/include</value>
<final>true</final>
</property>
<property>
<name>dfs.hosts.exclude</name>
<value>/usr/local/common/hadoop/conf/exclude</value>
<final>true</final>
</property>
<property>
<name>dfs.datanode.max.xcievers</name>
<value>8192</value>
</property>
<property>
<name>dfs.namenode.handler.count</name>
<value>128</value>
</property>
<property>
<name>dfs.datanode.handler.count</name>
<value>32</value>
</property>
<property>
<name>dfs.web.ugi</name>
<value>hadoop,supergroup</value>
</property>
<property>
<name>dfs.balance.bandwidthPerSec</name>
<value>52428800</value>
</property>

<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>zk4.mars.grid.sina.com.cn:2181,zk3.mars.grid.sina.com.cn:2181,zk2.mars.grid.sina.com.cn:2181,zk1.mars.grid.sina.com.cn:2181,zk5.mars.grid.sina.com.cn:2181</value>
</property>

<property>
<name>dfs.client.failover.proxy.provider.hbaseCluster</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>sshfence(hadoop:26387)</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.connect-timeout</name>
<value>10000</value>
</property>

<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/usr/home/hadoop/.ssh/id_rsa</value>
</property>
</configuration>

#配置core-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>


<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://hbaseCluster</value>
<description>缺省文件服务的协议和NS逻辑名称，和hdfs-site里的对应
此配置替代了1.0里的fs.default.name</description>
</property>

<property>
<name>fs.trash.interval</name>
<value>30</value>
<final>true</final>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/tmp/hadoop-${user.name}-${hue.suffix}</value>
<final>true</final>
</property>
<property>
<name>io.compression.codecs</name>
<value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.SnappyCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec</value>
<final>true</final>
</property>
<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>
</configuration>

#配置 slaves文件

hostname1

hostname2

hostname3

五 hadoop初始化

一定要配置/etc/hosts！

首先，配置自动ha后，需要先启动所有的journalnode，需要到各journalnode机器上执行：

hadoop-daemon.sh start journalnode

其次，hdfs namenode -format [<clusterID>]，在某一台namenode上执行即可，若另一台无法启动，则将集群停掉，将namenode的目录复制过去即可

再次,格式化ha的zk监控

$hdfs zkfc -formatZK
启动DFSZKFailoverController
hadoop-daemon.sh start zkfc

最后，启动HA，此处参考英文版的说明：

If you are setting up a fresh HDFS cluster, you should first run the format command (hdfs namenode -format) on one of NameNodes.
If you have already formatted the NameNode, or are converting a non-HA-enabled cluster to be HA-enabled, you should now copy over the contents of your NameNode metadata directories to the other, unformatted NameNode by running the command "hdfs namenode -bootstrapStandby" on the unformatted NameNode. Running this command will also ensure that the shared edits directory (as configured by dfs.namenode.shared.edits.dir) contains sufficient edits transactions to be able to start both NameNodes.
If you are converting a non-HA NameNode to be HA, you should run the command "hdfs namenode -initializeSharedEdits", which will initialize the shared edits directory with the edits data from the local NameNode edits directories.

最后启动全部进程：

$start-dfs.sh

六测试HA

kill active namenode，standby namenode 变成 active，耗费时间 3s