CDH6.2安装之YUM方式

参考:

https://www.sysit.cn/blog/post/sysit/CDH6.2.0%E7%B3%BB%E7%BB%9F%E9%83%A8%E7%BD%B2%E6%89%8B%E5%86%8C

https://www.cnblogs.com/yinzhengjie/articles/11029566.html

https://www.cnblogs.com/yinzhengjie/p/10372369.html

官网

https://www.cloudera.com/documentation/enterprise/6/6.2/topics/install_cm_cdh.html

https://blog.51cto.com/7072753/2176533?source=dra

1/ 配置cloudera官方源

每个节点

https://www.cloudera.com/documentation/enterprise/6/6.2/topics/configure_cm_repo.html

wget https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/cloudera-manager.repo
mv cloudera-manager.repo /etc/yum.repos.d/
rpm --import https://archive.cloudera.com/cm6/6.2.0/redhat7/yum/RPM-GPG-KEY-cloudera

2/ Installing Oracle JDK Using Cloudera Manager

#一般yum安装的oracle-jdk都放在/usr/java下：

yum install java -y

配置环境变量，在 profie 文 件 末 尾 添 加 jdk 路 径 ：
export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera

export CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

source /etc/profile

#一般open-jdk在/usr/lib/jvm/

参考: https://www.cnblogs.com/hongfeng2019/p/11331688.html

tips: /etc/profile注释掉的也会生效,删除老配置,source, exit用户再进入即生效.

3/ 安装cm

管理节点：
yum install cloudera-manager-daemons
yum install cloudera-manager-agent
yum install cloudera-manager-server
其它节点：
yum install cloudera-manager-agent

4/ 在管理节点mysql建相关库

# 登录mysql
mysql -uroot -p

建立库前：
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| test |
+--------------------+
5 rows in set (0.00 sec)

#数据库配置

https://www.cloudera.com/documentation/enterprise/6/6.2/topics/cm_ig_mysql.html#cmig_topic_5_5
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON scm.* TO 'scm'@'localhost' IDENTIFIED BY 'Fengfeng99~';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON amon.* TO 'amon'@'localhost' IDENTIFIED BY 'Fengfeng99~';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON rman.* TO 'rman'@'localhost' IDENTIFIED BY 'Fengfeng99~';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON hue.* TO 'hue'@'localhost' IDENTIFIED BY 'Fengfeng99~';
CREATE DATABASE hive DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hive.* TO 'hive'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON hive.* TO 'hive'@'localhost' IDENTIFIED BY 'Fengfeng99~';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON sentry.* TO 'sentry'@'localhost' IDENTIFIED BY 'Fengfeng99~';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON nav.* TO 'nav'@'localhost' IDENTIFIED BY 'Fengfeng99~';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON navms.* TO 'navms'@'localhost' IDENTIFIED BY 'Fengfeng99~';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'Fengfeng99~';
GRANT ALL ON oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'Fengfeng99~';

flush privileges;

#查看授权情况
SHOW GRANTS FOR 'hive'@'%';

#建库后：
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| amon |
| hue |
| metastore |
| mysql |
| nav |
| navms |
| oozie |
| performance_schema |
| rman |
| scm |
| sentry |
| sys |
| test |
+--------------------+

安装mysql-connector-java：
wget https://cdn.mysql.com//Downloads/Connector-J/mysql-connector-java-5.1.47.tar.gz
tar -zxvf mysql-connector-java-5.1.47.tar.gz
mkdir /usr/share/java
cp mysql-connector-java-5.1.47/mysql-connector-java-5.1.47-bin.jar /usr/share/java/mysql-connector-java.jar

初始化数据库：
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm Fengfeng99~

#如果数据库在另一台机器上用：https://www.cloudera.com/documentation/enterprise/6/6.2/topics/prepare_cm_database.html
/opt/cloudera/cm/schema/scm_prepare_database.sh mysql -h db01.example.com --scm-host cm01.example.com scm scm

#调整Cloudera Manager Server的默认堆内存大小，从2G调成4G
grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server
export CMF_JAVA_OPTS="-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"

#修改各节点的配置文件

grep server_port /etc/cloudera-scm-agent/config.ini #server_port 是Server和Agent的通信端口,别乱改
server_port=7182

grep server_host /etc/cloudera-scm-agent/config.ini #CM服务器默认是本机
server_host=localhost
#修改为指向node1
ansible client -m shell -a 'sed -i 's/server_host=localhost/server_host=zhep-opay-temp-1.novalocal/g' /etc/cloudera-scm-agent/config.ini'
ansible client -m shell -a 'grep server_host /etc/cloudera-scm-agent/config.ini'

# 启动
systemctl start cloudera-scm-server
比较慢,大概需要5-10分钟,可打开日志看,不断会更新数据库的sechma版本
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log
当看到这一句时,恭喜你,成功了.
2019-08-10 13:03:09,688 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.

systemctl enable cloudera-scm-server
systemctl status cloudera-scm-server
# 查看监听端口
ss -antlp | grep 7180
安装过程中会遇到的一些问题:
https://www.cloudera.com/documentation/enterprise/6/6.2/topics/cm_ig_troubleshooting.html#cmig_topic_19

管理节点和各节点:

systemctl restart cloudera-scm-agent
systemctl enable cloudera-scm-agent
systemctl status cloudera-scm-agent