编译过程漫长无比，错误百出，需要耐心耐心！！

1.准备的环境及软件

操作系统：Centos6.4 64位

jdk：jdk-7u80-linux-x64.rpm，不要使用1.8

maven：apache-maven-3.3.3-bin.tar.gz

protobuf：protobuf-2.5.0.tar.gz　　注：谷歌的产品，最好是提前百度准备一下这个文件

hadoop src：hadoop-2.5.2-src.tar.gz     hadoop的官网下载

ant：apache-ant-1.9.6-bin.tar.gz             编译common的时候会用到

openssl devel

ncurses-devel

CMake:2.6 or later

编译过程保证网络畅通，需要从外网下载其他的依赖包

2.环境准备过程

1）安装jdk

比较简单通过rpm -ivh jdk-7u80-linux-x64.rpm安装即可

2）解压hadoop源码包
tar zxvf hadoop-2.5.2-src.tar.gz

查看文件夹hadoop-2.5.2-src内的BUILDING.txt ，里面有编译所需要的环境要求

vi BUILDING.txt

3）安装maven（root用户安装）

直接解压缩到目录/usr

tar zxvf apache-maven-3.3.3-bin.tar.gz -C /usr

设置环境变量

vi + /etc/profile

export MAVEN_HOME=/usr/apache-maven-3.3.3

export PATH=.:$PATH:$MAVEN_HOME/bin

使用source命令使配置文件生效

source /etc/profile

验证maven是否安装成功mvn -version，显示maven，java，linux等信息

4）安装protobuf

解压缩到/usr目录

tar zxvf protobuf-2.5.0.tar.gz -C /usr

切换到目录/usr/protobuf-2.5.0中然后执行./configrue，make命令

./configure

需要安装gcc c++执行yum进行安装

yum install gcc

yum install gcc-c++  #会两次提示输入y（yes/no）

yum install make

我在进行上面第二个命令，对configure进行预编译时报错，查看异常后，是c++没有安装成功，便再一次执行了：yum install  gcc-c++ 命令，OK

安装完成后执行./configure通过后执行下面的make命令

make

make install

命令执行的时间比较长，最后一个命令的执行安装，如果出现错误，重新来过即可

执行完成后测试是否安装成功

protoc --version

5）安装CMake、openssl-devel

CMake 需要2.6以上的版本，在安装的时候会提示输入 y/N，同时由于安装这个组件是需要联网的，故根据网速不同，安装的进度也不一样

yum install cmake

yum install openssl-devel

yum install ncurses-devel
6）安装ant

解压文件

tar zxvf apache-ant-1.9.6-bin.tar.gz -C /usr

设置环境变量

vi + /etc/profile

export ANT_HOME=/usr/apache-ant-1.9.6

export PATH=.:$PATH:$ANT_HOME/bin

使用source命令使配置文件生效

source /etc/profile

验证是否安装成功ant -version，显示版本信息

3.编译hadoop源代码

切换为hadoop用户

切换到目录hadoop-2.5.2-src，执行命令

mvn package-DeskipTests -Pdist,native

之后是漫长的等待过程，中间过程会下载很多的文件，具体作用不详，经常因为下载网络问题失败，再次执行编译命令即可，之前下载过不会重新执行下载

出现失败情况，如果发现在输出[INFO]前的日志是停留在某个下载链接上，直接重试

出现编译测试错误，如下图，暂时未查明原因，直接重试一次

意外中断编译后再次开始后，编译异常，如下图

为了解决问题，通过以下命令重新尝试，分为两步操作

第一步、执行

mvn clean install –DskipTests

命令，等待完成(会自动联网下载很多东西）网络通畅后迅速搞定，不通畅的时候反复了多次

[INFO] ------------------------------------------------------------------------

[INFO] Reactor Summary:

[INFO]

[INFO] Apache Hadoop Main ................................. SUCCESS [  4.721 s]

[INFO] Apache Hadoop Project POM .......................... SUCCESS [  1.135 s]

[INFO] Apache Hadoop Annotations .......................... SUCCESS [  4.990 s]

[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  0.814 s]

[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.552 s]

[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  4.834 s]

[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  4.277 s]

[INFO] Apache Hadoop Auth ................................. SUCCESS [  5.709 s]

[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  2.516 s]

[INFO] Apache Hadoop Common ............................... SUCCESS [ 53.258 s]

[INFO] Apache Hadoop NFS .................................. SUCCESS [  1.175 s]

[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.070 s]

[INFO] Apache Hadoop HDFS ................................. SUCCESS [01:05 min]

[INFO] Apache Hadoop HttpFS ............................... SUCCESS [  4.430 s]

[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [ 28.926 s]

[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [  0.851 s]

[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.092 s]

[INFO] hadoop-yarn ........................................ SUCCESS [  0.063 s]

[INFO] hadoop-yarn-api .................................... SUCCESS [  4.983 s]

[INFO] hadoop-yarn-common ................................. SUCCESS [01:01 min]

[INFO] hadoop-yarn-server ................................. SUCCESS [  0.092 s]

[INFO] hadoop-yarn-server-common .......................... SUCCESS [ 20.415 s]

[INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [01:24 min]

[INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [  0.619 s]

[INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [  1.173 s]

[INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [  5.231 s]

[INFO] hadoop-yarn-server-tests ........................... SUCCESS [  1.075 s]

[INFO] hadoop-yarn-client ................................. SUCCESS [  1.853 s]

[INFO] hadoop-yarn-applications ........................... SUCCESS [  0.129 s]

[INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [  0.563 s]

[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [  0.515 s]

[INFO] hadoop-yarn-site ................................... SUCCESS [  0.112 s]

[INFO] hadoop-yarn-project ................................ SUCCESS [  0.132 s]

[INFO] hadoop-mapreduce-client ............................ SUCCESS [  0.151 s]

[INFO] hadoop-mapreduce-client-core ....................... SUCCESS [  8.638 s]

[INFO] hadoop-mapreduce-client-common ..................... SUCCESS [  5.152 s]

[INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [  1.165 s]

[INFO] hadoop-mapreduce-client-app ........................ SUCCESS [  4.623 s]

[INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [  1.714 s]

[INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [ 18.468 s]

[INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [  0.856 s]

[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [  1.955 s]

[INFO] hadoop-mapreduce ................................... SUCCESS [  0.096 s]

[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [  9.471 s]

[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [01:08 min]

[INFO] Apache Hadoop Archives ............................. SUCCESS [  0.841 s]

[INFO] Apache Hadoop Rumen ................................ SUCCESS [  1.113 s]

[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  2.258 s]

[INFO] Apache Hadoop Data Join ............................ SUCCESS [  0.981 s]

[INFO] Apache Hadoop Extras ............................... SUCCESS [  1.185 s]

[INFO] Apache Hadoop Pipes ................................ SUCCESS [  0.098 s]

[INFO] Apache Hadoop OpenStack support .................... SUCCESS [  1.246 s]

[INFO] Apache Hadoop Client ............................... SUCCESS [  0.427 s]

[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  0.336 s]

[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 21.003 s]

[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [  2.185 s]

[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.097 s]

[INFO] Apache Hadoop Distribution ......................... SUCCESS [  0.227 s]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 08:47 min

[INFO] Finished at: 2016-01-14T09:05:59+08:00

[INFO] Final Memory: 96M/237M

[INFO] ------------------------------------------------------------------------

执行编译

mvn package -Pdist,native -DskipTests -Dtar

编译成功

[INFO] Reactor Summary:

[INFO]

[INFO] Apache Hadoop Main ................................. SUCCESS [  6.143 s]

[INFO] Apache Hadoop Project POM .......................... SUCCESS [  3.923 s]

[INFO] Apache Hadoop Annotations .......................... SUCCESS [  6.345 s]

[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  0.466 s]

[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  3.457 s]

[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  8.004 s]

[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [  6.740 s]

[INFO] Apache Hadoop Auth ................................. SUCCESS [  7.214 s]

[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  6.145 s]

[INFO] Apache Hadoop Common ............................... SUCCESS [02:51 min]

[INFO] Apache Hadoop NFS .................................. SUCCESS [ 26.889 s]

[INFO] Apache Hadoop Common Project ....................... SUCCESS [  0.107 s]

[INFO] Apache Hadoop HDFS ................................. SUCCESS [04:44 min]

[INFO] Apache Hadoop HttpFS ............................... SUCCESS [04:57 min]

[INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [01:01 min]

[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 15.992 s]

[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [  0.307 s]

[INFO] hadoop-yarn ........................................ SUCCESS [  0.160 s]

[INFO] hadoop-yarn-api .................................... SUCCESS [02:30 min]

[INFO] hadoop-yarn-common ................................. SUCCESS [01:06 min]

[INFO] hadoop-yarn-server ................................. SUCCESS [  0.111 s]

[INFO] hadoop-yarn-server-common .......................... SUCCESS [ 27.082 s]

[INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [ 40.111 s]

[INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [  9.432 s]

[INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [ 15.236 s]

[INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 38.984 s]

[INFO] hadoop-yarn-server-tests ........................... SUCCESS [  0.889 s]

[INFO] hadoop-yarn-client ................................. SUCCESS [ 13.748 s]

[INFO] hadoop-yarn-applications ........................... SUCCESS [  0.165 s]

[INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [  7.001 s]

[INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [ 4.640 s]

[INFO] hadoop-yarn-site ................................... SUCCESS [  0.177 s]

[INFO] hadoop-yarn-project ................................ SUCCESS [ 13.099 s]

[INFO] hadoop-mapreduce-client ............................ SUCCESS [  0.141 s]

[INFO] hadoop-mapreduce-client-core ....................... SUCCESS [ 39.011 s]

[INFO] hadoop-mapreduce-client-common ..................... SUCCESS [ 33.940 s]

[INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [  7.769 s]

[INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 18.003 s]

[INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 16.896 s]

[INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [  7.284 s]

[INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [  3.377 s]

[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 11.164 s]

[INFO] hadoop-mapreduce ................................... SUCCESS [  5.954 s]

[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [  8.246 s]

[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 14.047 s]

[INFO] Apache Hadoop Archives ............................. SUCCESS [  3.657 s]

[INFO] Apache Hadoop Rumen ................................ SUCCESS [ 12.067 s]

[INFO] Apache Hadoop Gridmix .............................. SUCCESS [  8.998 s]

[INFO] Apache Hadoop Data Join ............................ SUCCESS [  5.860 s]

[INFO] Apache Hadoop Extras ............................... SUCCESS [  5.582 s]

[INFO] Apache Hadoop Pipes ................................ SUCCESS [ 11.003 s]

[INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 11.071 s]

[INFO] Apache Hadoop Client ............................... SUCCESS [ 16.352 s]

[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [  0.241 s]

[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [  8.179 s]

[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [  7.388 s]

[INFO] Apache Hadoop Tools ................................ SUCCESS [  0.048 s]

[INFO] Apache Hadoop Distribution ......................... SUCCESS [01:07 min]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 27:05 min

[INFO] Finished at: 2016-01-14T09:56:35+08:00

[INFO] Final Memory: 77M/237M

[INFO] ------------------------------------------------------------------------

检查编译的本地文件，完成编译

hadoop 2.5.2源码编译的更多相关文章

  1. 基于cdh5.10.x hadoop版本的apache源码编译安装spark

    参考文档:http://spark.apache.org/docs/1.6.0/building-spark.html spark安装需要选择源码编译方式进行安装部署,cdh5.10.0提供默认的二进 ...

  2. Hadoop（一）Hadoop的简介与源码编译

    一 Hadoop简介 1.1Hadoop产生的背景 1. HADOOP最早起源于Nutch.Nutch的设计目标是构建一个大型的全网搜索引擎,包括网页抓取.索引.查询等功能,但随着抓取网页数量的增加, ...

  3. hadoop 2.7.3 源码编译教程

    1.工具准备,最靠谱的是hadoop说明文档里要求具备的那些工具. 到hadoop官网,点击source下载hadoop-2.7.3-src.tar.gz. 解压之 tar -zxvf hadoop- ...

  4. Hadoop源码编译过程

    一.           为什么要编译Hadoop源码 Hadoop是使用Java语言开发的,但是有一些需求和操作并不适合使用java,所以就引入了本地库(Native Libraries)的概念,通 ...

  5. Hadoop,HBase,Zookeeper源码编译并导入eclipse

    基本理念:尽可能的参考官方英文文档 Hadoop:  http://wiki.apache.org/hadoop/FrontPage HBase:  http://hbase.apache.org/b ...

  6. hadoop 源码编译

    hadoop 源码编译 1.准备jar 1) hadoop-2.7.2-src.tar.gz 2) jdk-8u144-linux-x64.tar.gz 3) apach-ant-1.9.9-bin. ...

  7. Spark源码编译

    原创文章,转载请注明: 转载自http://www.cnblogs.com/tovin/p/3822995.html spark源码编译步骤如下: cd /home/hdpusr/workspace ...

  8. hadoop-1.2.0源码编译

    以下为在CentOS-6.4下hadoop-1.2.0源码编译步骤. 1. 安装并且配置ant 下载ant,将ant目录下的bin文件夹加入到PATH变量中. 2. 安装git,安装autoconf, ...

  9. hadoop-2.0.0-mr1-cdh4.2.0源码编译总结

    准备编译hadoop-2.0.0-mr1-cdh4.2.0的同学们要谨慎了.首先看一下这篇文章: Hadoop作业提交多种方案 http://www.blogjava.net/dragonHadoop ...

随机推荐

  1. 22_2mybatis——CURD

    1.CURD操作 第一步:创建maven工程并导入坐标 <?xml version="1.0" encoding="UTF-8"?> <pro ...

  2. LCA统计

    读入挂 inline void read(int &v) { v = ; ; ; ') { if (c == '-') { p = -; } c = getchar(); } ') { v = ...

  3. spring事务配置，声明式事务管理和基于@Transactional注解的使用（转载）

    原文地址:http://blog.csdn.net/bao19901210/article/details/41724355 事务管理对于企业应用来说是至关重要的,好使出现异常情况,它也可以保证数据的 ...

  4. zabbix命令之：zabbix_sender命令

    zabbix server除了可以从客户端主动获取数据,客户端也可以主动将数据推送给服务端,客户端通过zabbix_sender指令来实现向服务端主动推送数据. 在zabbix客户端安装 配置yum源 ...

  5. Kendo UI使用教程：CDN服务

    [Kendo UI最新试用版下载] Kendo UI目前最新提供Kendo UI for jQuery.Kendo UI for Angular.Kendo UI Support for React和 ...

  6. APIview的请求生命周期源码分析

    目录 APIview的请求生命周期源码分析 请求模块 解析模块 全局配置解析器 局部配置解析器 响应模块 异常处理模块 重写异常处理函数 渲染模块 APIview的请求生命周期源码分析 Django项 ...

  7. 导出csv文件(使用a标签)

    https://blog.csdn.net/oscar999/article/details/16342699   productsCSV(e) { const { download } = this ...

  8. [NOIP2012] 同余方程(第三次考试大整理)

    1265. [NOIP2012] 同余方程 输入文件:mod.in   输出文件:mod.out   简单对比 时间限制:1 s   内存限制:128 MB [题目描述] 求关于 x 的同余方程 ax ...

  9. OI多项式 简单学习笔记

    咕咕咕 先开个坑(其实是存模板来了) 一些特别简单的前置东西qwq 复数的计算 复数相加:向量相加,复数相乘.复数相乘:模长相乘,旋转量相加(就是复平面坐标轴逆时针旋转的角度) (当然也可以直接使用c ...

  10. 启用Executor初始化线程池

    前言 上文我们介绍了JDK中的线程池框架Executor.我们知道,只要需要创建线程的情况下,即使是在单线程模式下,我们也要尽量使用Executor.即: ExecutorService fixedT ...