ambari 2.5.0源码编译安装
参考：https://www.ibm.com/developerworks/cn/opensource/os-cn-bigdata-ambari/index.html
Ambari 是什么
Ambari 的作用来就是创建、管理、监视 Hadoop 集群，但是这里的 Hadoop 是广义，指的是 Hadoop 整个生态圈（例如 Hive，Hbase，Sqoop，Zookeeper 等），而并不仅是特指 Hadoop。用一句话来说，Ambari 就是为了让 Hadoop 以及相关的大数据软件更容易使用的一个工具。
Ambari 自身也是一个分布式架构的软件，主要由两部分组成：Ambari Server 和 Ambari Agent。简单来说，用户通过 Ambari Server 通知 Ambari Agent 安装对应的软件；Agent 会定时地发送各个机器每个软件模块的状态给 Ambari Server，最终这些状态信息会呈现在 Ambari 的 GUI，方便用户了解到集群的各种状态，并进行相应的维护
Ambari 2.5.0源码编译安装
三台机器(centeros)分别是m1、m2、m3，m1为Ambari Server，m2、m3为Ambari Agent
Ambari 2.5.0源码编译安装
三台机器(centeros)分别是m1、m2、m3，m1为Ambari Server，m2、m3为Ambari Agent
准备工作：
1、SSH 的无密码登录
Ambari 的 Server 会 SSH 到 Agent 的机器，拷贝并执行一些命令。因此我们需要配置 Ambari Server 到 Agent 的 SSH 无密码登录。在这里就是，m1可以 SSH 无密码登录 m2 和 m3
2、确保 Yum 可以正常工作
通过公共库（public repository），安装 Hadoop 这些软件，背后其实就是应用 Yum 在安装公共库里面的 rpm 包。所以这里需要您的机器都能访问 Internet。
3、确保机器的 Python 版本大于或等于 2.6
4、安装好maven
参考官网：https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.5.0
pushd命令常用于将目录加入到栈中，加入记录到目录栈顶部，并切换到该目录；若pushd命令不加任何参数，则会将位于记录栈最上面的2个目录对换位置
popd用于删除目录栈中的记录；如果popd命令不加任何参数，则会先删除目录栈最上面的记录，然后切换到删除过后的目录栈中的最上面的目录
预先安装的软件
参考：https://cwiki.apache.org/confluence/display/AMBARI/Ambari+Development
JDK 7
Apache Maven 3.3.x
Python 2.6
rpmbuild
g++ (gcc-c++ package)
ant
python-devel
git
psutils
setuptools
第一步：下载编译 Ambari 2.5.0源码
wget http://www.apache.org/dist/ambari/ambari-2.5.0/apache-ambari-2.5.0-src.tar.gz
tar xfvz apache-ambari-2.5.0-src.tar.gz
cd apache-ambari-2.5.0-src
mvn versions:set -DnewVersion=2.5.0.0.0
pushd ambari-metrics
mvn versions:set -DnewVersion=2.5.0.0.0
popd mvn -B clean install package rpm:rpm -DnewVersion=2.5.0.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true
第一步常见的错误
[ERROR] Failed to execute goal on project ambari-metrics-kafka-sink: Could not resolve dependencies for project org.apache.ambari:ambari-metrics-kafka-sink:jar:2.2.2-0: The following artifacts could not be resolved: javax.jms:jms:jar:1.1, com.sun.jdmk:jmxtools:jar:1.2.1, com.sun.jmx:jmxri:jar:1.2.1: Could not transfer artifact javax.jms:jms:jar:1.1 from/to java.net (https://maven-repository.dev.java.net/nonav/repository): No connector available to access repository java.net (https://maven-repository.dev.java.net/nonav/repository) of type legacy using the available factories WagonRepositoryConnectorFactory
解决方法
jmxri-1.2.1.pom
<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>javax.jms</groupId>
<artifactId>jms</artifactId>
<version>1.1</version>
<name>Java Message Service</name>
<description>
The Java Message Service (JMS) API is a messaging standard that allows application components based on the Java 2 Platform, Enterprise Edition (J2EE) to create, send, receive, and read messages. It enables distributed communication that is loosely coupled, reliable, and asynchronous.
</description>
<url>http://java.sun.com/products/jms</url>
<distributionManagement>
<downloadUrl>http://java.sun.com/products/jms/docs.html</downloadUrl>
</distributionManagement>
</project>
jmxri-1.2.1.pom
<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.sun.jmx</groupId>
<artifactId>jmxri</artifactId>
<version>1.2.1</version>
<distributionManagement>
<status>deployed</status>
</distributionManagement>
</project>
jmxtools-1.2.1.pom
<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.sun.jdmk</groupId>
<artifactId>jmxtools</artifactId>
<version>1.2.1</version>
<distributionManagement>
<status>deployed</status>
</distributionManagement>
</project>
mvn install:install-file -Dfile=jms-1.1.pom -DgroupId=javax.jms -DartifactId=jms -Dversion=1.1 -Dpackaging=jar mvn install:install-file -Dfile=jmxri-1.2.1.pom -DgroupId=com.sun.jmx -DartifactId=jmxri -Dversion=1.2.1 -Dpackaging=jar mvn install:install-file -Dfile=jmxtools-1.2.1.pom -DgroupId=com.sun.jdmk -DartifactId=jmxtools -Dversion=1.2.1 -Dpackaging=jar
错误
Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (default) on project ambari: Too many files with unapproved license: 3 See RAT report in: /home/hadoop/apache-ambari-2.5.0-src/target/rat.txt -> [Help 1]
mvn -B clean install package rpm:rpm -DnewVersion=2.5.0.0.0 -DskipTests -Dpython.ver="python >= 2.6" -Drat.skip=true -DskipTests
添加：-Drat.skip=true
Failed to execute goal org.codehaus.mojo:rpm-maven-plugin:2.0.1:rpm (default-cli) on project ambari: RPM build execution returned: '127' executing '/bin/sh -c cd /home/hadoop/apache-ambari-2.5.0-src/target/rpm/ambari/SPECS && rpmbuild -bb --buildroot /home/hadoop/apache-ambari-2.5.0-src/target/rpm/ambari/buildroot --define '_topdir /home/hadoop/apache-ambari-2.5.0-src/target/rpm/ambari' --target noarch-redhat-linux ambari.spec' -> [Help 1]
原因是rpm或者rpm-build不存在
安装 rpm rpm-build
yum install rpm
yum install -y rpm-build.x86_64
[ERROR] Failed to execute goal on project ambari-metrics-storm-sink: Could not resolve dependencies for project org.apache.ambari:ambari-metrics-storm-sink:jar:2.5.0.0.0: Could not find artifact org.apache.storm:storm-core:jar:1.1.0-SNAPSHOT in alimaven (http://maven.aliyun.com/nexus/content/groups/public/) -> [Help 1]
这是因为org.apache.storm:storm-core:jar:1.1.0-SNAPSHOT不存在导致的问题
修改/apache-ambari-2.5.0-src/ambari-metrics/ambari-metrics-storm-sink/pox.xml修改storm为1.1.0版本
[INFO] Downloaded: http://maven.aliyun.com/nexus/content/groups/public/org/apache/commons/commons-exec/1.3/commons-exec-1.3.jar (54 kB at 168 kB/s)
bower angular-mocks#1.2.26 ENOGIT git is not installed or not in the PATH
[ERROR] Command execution failed.
没有安装git，安装git即可
yum -y install git
[ERROR] around Ant part ...<get dest="/home/hadoop/apache-ambari-2.5.0-src/ambari-metrics/ambari-metrics-timelineservice/target/embedded/phoenix.tar.gz" src="http://private-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.4.0-3347/tars/phoenix-4.4.0.2.3.4.0-3347.tar.gz" usetimestamp="true"/>... @ 5:270 in /home/hadoop/apache-ambari-2.5.0-src/ambari-metrics/ambari-metrics-timelineservice/target/antrun/build-Download Phoenix.xml
没有安装ant，安装ant
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (psutils-compile) on project ambari-metrics-host-monitoring: An Ant BuildException has occured: exec returned: 1
[ERROR] around Ant part ...<exec dir="/home/hadoop/apache-ambari-2.5.0-src/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/psutil" executable="/home/hadoop/apache-ambari-2.5.0-src/ambari-metrics/ambari-metrics-host-monitoring/../../ambari-common/src/main/unix/ambari-python-wrap" failonerror="true">... @ 4:289 in /home/hadoop/apache-ambari-2.5.0-src/ambari-metrics/ambari-metrics-host-monitoring/target/antrun/build-psutils-compile.xml
缺少python psutils模块，安装如下
wget https://pypi.python.org/packages/source/p/psutil/psutil-2.0.0.tar.gz
tar -xzvf psutil-2.0.0.tar.gz
cd psutil-2.0.0
python setup.py install
安装python setup.py install时出现如下错误
psutil/_psutil_linux.c:12:20: error: Python.h: No such file or directory
需要安装
yum -y install python-devel
配置Maven内存
Linux
export MAVEN_OPTS="-Xms512m -Xmx1024m -XX:MaxPermSize=512m"
windows
set MAVEN_OPTS=-Xms512m -Xmx1024m -XX:MaxPermSize=512m
[INFO] --- exec-maven-plugin:1.2.1:exec (python-package) @ python-client ---
Traceback (most recent call last):
File "/home/hadoop/apache-ambari-2.5.0-src/ambari-client/python-client/src/main/python/setup.py", line 18, in <module>
from setuptools import setup, find_packages
ImportError: No module named setuptools
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-package) on project python-client: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (python-package) on project python-client: Command execution failed.
setuptools安装
http://blog.chinaunix.net/uid-26000296-id-4769414.html
编译ambari-metrics时比较慢，主要是因为里面包含的几个文件下载时间太长，可以修改apache-ambari-2.5.0-src/ambari-metrics/pom.xml中properties属性下的文件下载地址为本地，然后，通过迅雷下载，如下
http://private-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.4.0-3347/tars/hbase-1.1.2.2.3.4.0-3347.tar.gz
http://private-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.3.4.0-3347/tars/hadoop-2.7.1.2.3.4.0-3347.tar.gz
https://msibuilds.blob.core.windows.net/hdp/2.x/2.2.4.2/2/hbase-0.98.4.2.2.4.2-0002-hadoop2.winpkg.zip
https://msibuilds.blob.core.windows.net/hdp/2.x/2.2.4.2/2/hadoop-2.6.0.2.2.4.2-0002.winpkg.zip
https://grafanarel.s3.amazonaws.com/builds/grafana-2.6.0.linux-x64.tar.gz
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<python.ver>python >= 2.6</python.ver>
<deb.python.ver>python (>= 2.6)</deb.python.ver>
<!--TODO change to HDP URL-->
<hbase.tar>file:///home/hadoop/apache-ambari-2.5.0-src/tars/hbase-1.1.2.2.3.4.0-3347.tar.gz</hbase.tar>
<hbase.folder>hbase-1.1.2.2.3.4.0-3347</hbase.folder>
<hadoop.tar>file:///home/hadoop/apache-ambari-2.5.0-src/tars/hadoop-2.7.1.2.3.4.0-3347.tar.gz</hadoop.tar>
<hadoop.folder>hadoop-2.7.1.2.3.4.0-3347</hadoop.folder>
<hbase.winpkg.zip>file:///home/hadoop/apache-ambari-2.5.0-src/tars/hbase-0.98.4.2.2.4.2-0002-hadoop2.winpkg.zip</hbase.winpkg.zip>
<hbase.winpkg.folder>hbase-0.98.4.2.2.4.2-0002-hadoop2</hbase.winpkg.folder>
<hadoop.winpkg.zip>file:///home/hadoop/apache-ambari-2.5.0-src/tars/hadoop-2.6.0.2.2.4.2-0002.winpkg.zip</hadoop.winpkg.zip>
<hadoop.winpkg.folder>hadoop-2.6.0.2.2.4.2-0002</hadoop.winpkg.folder>
<grafana.folder>grafana-2.6.0</grafana.folder>
<grafana.tar>file:///home/hadoop/apache-ambari-2.5.0-src/tars/grafana-2.6.0.linux-x64.tar.gz</grafana.tar>
<phoenix.tar>file:///home/hadoop/apache-ambari-2.5.0-src/tars/phoenix-4.4.0.2.3.4.0-3347.tar.gz</phoenix.tar>
<phoenix.folder>phoenix-4.4.0.2.3.4.0-3347</phoenix.folder>
<resmonitor.install.dir>
/usr/lib/python2.6/site-packages/resource_monitoring
</resmonitor.install.dir>
<powermock.version>1.6.2</powermock.version>
<distMgmtSnapshotsId>apache.snapshots.https</distMgmtSnapshotsId>
<distMgmtSnapshotsName>Apache Development Snapshot Repository</distMgmtSnapshotsName>
<distMgmtSnapshotsUrl>https://repository.apache.org/content/repositories/snapshots</distMgmtSnapshotsUrl>
<distMgmtStagingId>apache.staging.https</distMgmtStagingId>
<distMgmtStagingName>Apache Release Distribution Repository</distMgmtStagingName>
<distMgmtStagingUrl>https://repository.apache.org/service/local/staging/deploy/maven2</distMgmtStagingUrl>
</properties>
第二步: 安装 Ambari Server
cd apache-ambari-2.5.0-src/ambari-server/target/rpm/ambari-server/RPMS/x86_64 yum -y install ambari-server*.rpm
第三步：设置启动ambari-server（使用root进行设置和启动）
安装Mysql
yum install -y mysql-server
配置mysql数据库
启动Mysql：
service mysqld start
连接数据库：
mysql -u root -p
create database ambari;
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
mysqladmin -u root password "root"
mysql -u root -p直接回车，此时不需要输入密码
ambari-server setup
/usr/sbin/ambari-server: line 34: buildNumber: unbound variable
脚本/usr/sbin/ambari-server中的变量buildNumber没有定义
export buildNumber=2.5.0.0
ambari-server setup
WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java and set property "server.jdbc.driver.path=[path/to/custom_jdbc_driver]" in ambari.properties.
Press <enter> to continue.
拷贝/usr/share/java，设置ambari.properties server.jdbc.driver.path=[path/to/custom_jdbc_driver]
server.jdbc.driver.path=/usr/share/java/mysql-connector-java-5.1.42-bin.jar
ambari.properties文件在/etc/ambari-server/conf下
配置好后需要在重新的ambari-server setup一次，不然启动是不成功
ambari-server start
启动日志
/var/log/ambari-server/ambari-server-command.log
WARNING: The hostname was not found in the reverse DNS lookup. This may result in incorrect behavior
主机名有问题，在hosts文件中配置一下就好了
启动成功应该如下
[root@bogon ambari-server]# ambari-server start
Using python /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start...........................
Server started listening on 8080 DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.
web访问地址
http://192.168.1.105:8080/
用户名和密码都是admin
配置私钥的时候红色的部分也要一起拷贝
上面的错误是因为其他节点没有安装ambari-agent
ambari 2.5.0源码编译安装的更多相关文章
- Impala 3.3.0 源码编译安装
最新编译Apache-impala 的心酸历程.大概花了10天才整好,极度的崩溃!!!由于国内的上网环境大家都懂的,访问国外的s3.amazonaws.com一些资源库的速度极其感人,尤其 ...
- 源码编译安装MySQL8.0
源码编译安装MySQL8.0 0.前期准备条件 查看linux的版本 [root@mysql etc]# cat /etc/redhat-release CentOS Linux release 7. ...
- 保姆级教程——Ubuntu16.04 Server下深度学习环境搭建：安装CUDA8.0，cuDNN6.0，Bazel0.5.4，源码编译安装TensorFlow1.4.0(GPU版)
写在前面 本文叙述了在Ubuntu16.04 Server下安装CUDA8.0,cuDNN6.0以及源码编译安装TensorFlow1.4.0(GPU版)的亲身经历,包括遇到的问题及解决办法,也有一些 ...
- [笔记] Ubuntu 18.04源码编译安装OpenCV 4.0流程
标准常规安装方法安装的OpenCV版本比较低,想尝鲜使用4.0版本,只好源码安装. 安装环境 OS:Ubuntu 18.04 64 bit 显卡:NVidia GTX 1080 CUDA:10.0 c ...
- centos 7 下 rabbitmq 3.8.0 & erlang 22.1 源码编译安装
centos 7 下 rabbitmq 3.8.0 & erlang 22.1 源码编译安装 安装前请检查好erlang和rabbitmq版本是否相匹配参考:RabbitMQ Erlang V ...
- centos7.6环境zabbix3.2源码编译安装版升级到zabbix4.0长期支持版
zabbix3.2源码编译安装版升级到zabbix4.0长期支持版 项目需求: .2版本不再支持,想升级成4.0的长期支持版 环境介绍: zabbix服务端是编译安装的,数据库和web在一台机器上 整 ...
- 源码编译安装MySQL8.0.20
1 概述 本文章主要讲述了如何从源码编译安装MySQL社区版8.0.20,首先会介绍一些编译安装的相关知识,然后开始编译安装 2 源码编译安装的相关知识 2.1 make与configure make ...
- SSH/SSL 源码编译安装简易操作说明
环境:CentOS 6.7 安全加固需求,由于某盟扫描系统主机有SSL系列漏洞,客户要求必须修复: 解决方案:将SSH/SSL升级到最新版本,删除SSL旧版本(实测不删除旧版本某盟扫描无法通过). 当 ...
- 源码编译安装 MySQL 5.5.x 实践
1.安装cmakeMySQL从5.5版本开始,通过./configure进行编译配置方式已经被取消,取而代之的是cmake工具.因此,我们首先要在系统中源码编译安装cmake工具. # wget ht ...
随机推荐
- Tire树模板-于是他错误的点名开始了
题目背景 XS中学化学竞赛组教练是一个酷爱炉石的人. 他会一边搓炉石一边点名以至于有一天他连续点到了某个同学两次,然后正好被路过的校长发现了然后就是一顿欧拉欧拉欧拉(详情请见已结束比赛CON900). ...
- Java基础之文件的输入输出流操作
在介绍输入输出流之前,首先需要了解如何创建文件,创建文件夹以及遍历文件夹等各种操作,这里面不在一一介绍,主要介绍的是文件的输入输出流操作. 在起初学习文件操作之前,总是喜欢将输入输出弄混淆,后来通过看 ...
- 二分类算法的评价指标：准确率、精准率、召回率、混淆矩阵、AUC
评价指标是针对同样的数据,输入不同的算法,或者输入相同的算法但参数不同而给出这个算法或者参数好坏的定量指标. 以下为了方便讲解,都以二分类问题为前提进行介绍,其实多分类问题下这些概念都可以得到推广. ...
- JS中keyup, keypress, keydown以及oninput四个事件的区别
$email_input.onkeyup=function(event){ // console.log('event handle');//按方向键以及backspce esc有反应 长按字母键也没 ...
- centOS7 flask项目布署
先用1张图表示centOS布署flask的关键点,以及可能遇到的问题,及解决办法. 图片看不清,可以[下载]下来看,清晰度能够看清字 [目标] 局域网内,通过url可以访问flask编写的api 更新 ...
- 阶段3 1.Mybatis_06.使用Mybatis完成DAO层的开发_4 Mybatis中使用Dao实现类的执行过程分析-查询方法
delete方法没有并SqlSession的delete方法,而是调用的Upadte方法. 在测试类这里加断点. 实际的方法体内也加断点 运行测试方法,选择debug的方式 走到断点这里.会看到fac ...
- Jmeter之检查点
检查点 Jmeter中检查点也叫断言,Jmeter中有很多类型的断言,但是最常用的是响应断言,即根据服务器返回的结果来判断测试是否通过. 响应断言 添加断言结果用于运行完毕后查看结果 测试通过 测试不 ...
- wpf 非窗体类中 异步调用窗体与控件
App.Current.Dispatcher.Invoke((Action)(() => { MessageBoxWindow mwb = ); mwb.ShowDialog(); return ...
- mysql源码包安装
一.准备编译环境 # yum -y install ncurses ncurses-devel openssl-devel bison gcc gcc-c++ make cmake # wget ht ...
- 应用安全 - CMS - Discuz漏洞汇总
SSV-90861 Date:2012 类型:敏感信息泄露 影响范围:DZ x2.5 POC:http://www.xx.xx/uc_server/control/admin/db.php http ...