RedHat Ent 6.5 64bit编译安装hadoop2.4.1

RedHat Ent 6.5 64bit编译安装hadoop2.4.1

感谢原帖：http://blog.csdn.net/w13770269691/article/details/16883663/

step 1.修改yum：（针对redhat ent未注册用户，注册用户直接跳过这一步）

参考：http://blog.csdn.net/zhngjan/article/details/20843465

step 2.下载源码包：http://mirrors.hust.edu.cn/apache/hadoop/common/stable2/

wget http://mirrors.hust.edu.cn/apache/hadoop/common/stable2/hadoop-2.4.1-src.tar.gz

step 3.准备maven作为编译hadoop的工具

a.下载编译好的maven包：

wget http://mirror.bit.edu.cn/apache/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz
tar -zxvf apache-maven-3.1.1-bin.tar.gz -C /opt/

b.配置maven的环境变量，在/etc/profile文件结尾中添加如下代码

export MAVEN_HOME=/opt/apache-maven-3.1.
export PATH=$PATH:${MAVEN_HOME}/bin

c.执行如下命令使配置文件生效

source /etc/profile

d.测试maven

mvn -version

e.由于maven国外服务器可能连不上，先给maven配置一下国内镜像

在maven目录下，conf/settings.xml,在<mirrors></mirros>里添加如下内容（注意不要添加到注释里面了）

<mirror>
  <id>nexus-osc</id>
  <mirrorOf>*</mirrorOf>
  <name>Nexusosc</name>
  <url>http://maven.oschina.net/content/groups/public/</url>
</mirror>

在maven目录下，conf/settings.xml,在<profiles></profiles>添加如下内容（注意不要添加到注释里面了）

<profile>
  <id>jdk-1.7</id>
  <activation>
  <jdk>1.7</jdk>
  </activation>
  <repositories>
  <repository>
   <id>nexus</id>
   <name>local private nexus</name>
   <url>http://maven.oschina.net/content/groups/public/</url>
   <releases>
    <enabled>true</enabled>
   </releases>
   <snapshots>
    <enabled>false</enabled>
   </snapshots>
  </repository>
  </repositories>
  <pluginRepositories>
   <pluginRepository>
    <id>nexus</id>
    <name>local private nexus</name>
    <url>http://maven.oschina.net/content/groups/public/</url>
    <releases>
      <enabled>true</enabled>
    </releases>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
   </pluginRepository>
  </pluginRepositories>
</profile>  

step 4.hadoop2.4.1编译需要protoc2.5.0的支持，所以还要安装下载protoc2.5.0

官方网址：https://code.google.com/p/protobuf/downloads/list

百度网盘网址：http://pan.baidu.com/s/1pJlZubT

a.对protoc进行编译安装前先要装几个依赖包:gcc,gcc-c++,make 如果已经安装的可以忽略

yum install gcc
yum intall gcc-c++
yum install make 

b.配置protoc

tar -xvf protobuf-2.5..tar.bz2
cd protobuf-2.5.
./configure --prefix=/opt/protoc/
make && make install
yum install gcc 

红色字体是自定义的protoc安装目录，完成上面的命令后，再配置protoc环境变量，同样在/etc/profile尾部加入：

export PATH=/opt/protoc/bin:$PATH

step 5.，还不要着急开始编译安装，不然又是各种错误,需要安装cmake,openssl-devel,ncurses-devel依赖

yum install cmake
yum install openssl-devel
yum install ncurses-devel  

step 6. 解压并编译hadoop源码(一定要进入到源码解压的位置再执行mvn编译)

tar -zxvf hadoop-2.4.1-src.tar.gz -C /opt/
cd /opt/hadoop-2.4.1-src
mvn package -Pdist,native -DskipTests -Dtar

编译耗时比较长，耐心等待，成功后结果类似于：

 pache Hadoop Distributed Copy .................... SUCCESS [.648s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [.303s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [.288s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [.611s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [.334s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [.737s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [.321s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [.136s]
[INFO] Apache Hadoop Client .............................. SUCCESS [.563s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [.254s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [.245s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [.478s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [.084s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [.979s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: :.464s
[INFO] Finished at: Thu Aug  :: CST
[INFO] Final Memory: 159M/814M
[INFO] ------------------------------------------------------------------------
You have new mail in /var/spool/mail/root

编译后的路径在:hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1

进入hadoop-2.4.1目录，测试安装是否成功

[root@cupcs-redhat6 bin]# cd /opt/hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1/bin
[root@cupcs-redhat6 bin]# ./hadoop version
Hadoop 2.4.1
Subversion Unknown -r Unknown
Compiled by root on 2014-08-07T05:52Z
Compiled with protoc 2.5.0
From source with checksum bb7ac0a3c73dc131f4844b873c74b630
This command was run using
/opt/hadoop-2.4.1-src/hadoop-dist/target/hadoop-2.4.1/share/hadoop/common/hadoop-common-2.4.1.jar
[root@cupcs-redhat6 bin]# cd ..
[root@cupcs-redhat6 hadoop-2.4.1]# file lib/native/*
lib/native/libhadoop.a: current ar archive
lib/native/libhadooppipes.a: current ar archive
lib/native/libhadoop.so: symbolic link to `libhadoop.so.1.0.0'
lib/native/libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1
(SYSV), dynamically linked, not stripped
lib/native/libhadooputils.a: current ar archive
lib/native/libhdfs.a: current ar archive
lib/native/libhdfs.so: symbolic link to `libhdfs.so.0.0.0'
lib/native/libhdfs.so.0.0.0: ELF 64-bit LSB shared object, x86-64, version 1
(SYSV), dynamically linked, not stripped
[root@cupcs-redhat6 hadoop-2.4.1]#