其实,基本思路跟如下差不多,我就不多详细说了,贴出主要图。

  博主,我是直接借鉴下面这位博主,来进行安装的!(灰常感谢他们!)

在线和离线安装Cloudera CDH 5.6.0

 Cloudera Manager 5和CDH5离线安装

Ubuntu14.04用apt在线/离线安装CDH5.1.2[Apache Hadoop 2.3.0

https://www.zybuluo.com/ncepuwanghui/note/474966

 ClouderManager官网

https://www.cloudera.com/documentation/manager/5-1-x/Cloudera-Manager-Installation-Guide/cm5ig_mysql.html#id_trn_pps_xm_unique_1

说在前面的话

  我的机器是总共4台,分别为ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

  在ubuntucmbigdata1上执行了

sudo apt-get install cloudera-manager-daemons cloudera-manager-server 

sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent 

  即,ubuntucmbigdata1是既做server,又做agent。

  在ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4上执行了

sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent 

  即,ubuntucmbigdata2、3、和4是只做agent。

  在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list,复制以下内容到cloudera.list里

# Packages for Cloudera Manager, Version , on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

  并且在在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

  向apt添加cloudera源公钥,在所有节点执行如下命令:

curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

然后,再

sudo apt-get update

  具体在后面我会详细教大家怎么安装。

  

  的确,我也跟这位博主一样,也需要在实验室里部署。以及也是4台机器。(至于原因,大家都懂得,中国高校研究生喜欢4、3、2、1来进行加速比做实验论文)

在实验室安装过社区版的Hadoop,当时用的是普通PC机,安装的过程遇到了好多问题。当解决了大部分的安装问题之后由于实验室的需求,需要安装一个商业发行版的hadoop ,我们就采用的Cloudera 公司发行的版本。首先要说明的是CDH对内存要求比较高,当我们采用原来的4G内存的PC机,只要启动基本内存就满了,并且会拖慢整个集群的运算速度。所以说安装CDH对硬件的要求比较高。安装CDH 有三种安装方式:在线安装、半离线安装、离线安装。在线安装和离线安装我都安装过,虽然在线安装貌似简单但是实际用起来还是没有离线安装方便,我还是建议大家采用离线安装的方式。不过下面都会介绍在线安装的方式和离线安装的方式。希望对别人有所帮助

  

  其它的,ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

  

 参考 第一步:

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5)(一)

Ubuntu14.04安装之后的一些配置

参考

ubuntu 下 visudo的使用

  在文件末尾(按快捷键,Shift + g)添加bigdata  ALL=(ALL) NOPASSWD:ALL

ctrl+o 保存

ctrl+x 退出。

  最后,得到如下

  我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

  防火墙

  参考

hadoop-2.6.0.tar.gz + spark-1.5.2-bin-hadoop2.6.tgz的集群搭建(单节点)(Ubuntu系统)

 

  由此,表明Ubuntu14.04是默认没开启防火墙的。

 我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

相关服务的安装

  如果没有安装的话可以使用下面的命令安装一下。

  参考

Ubuntu 14.04 安装 Cloudera Manager 5 And CDH 5 (在线安装)

bigdata@ubuntucmbigdata1:~$ sudo apt-get install lrzsz

bigdata@ubuntucmbigdata1:~$ sudo apt-get install openssh-server 

  这个工具要装(你懂得),2个理由:1,可以远程管理系统。2,Cloudera Manages 集群管理需要用该服务。

 我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

ssh 服务配置、vim的安装

  参考

Ubuntu14.04安装之后的一些配置

  打开sshd_config文件, 在  /etc/ssh 目录下。

# Authentication:
LoginGraceTime 120
#PermitRootLogin without-password
PermitRootLogin yes
StrictModes yes

  注释掉 #PermitRootLogin without-password 行。

  添加 PermitRootLogin yes 行

  修改后,如下:

 我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

修改hosts文件

  vi /etc/hosts

在里面添加主机名和IP之间的对应关系

#127.0.1.1      ubuntucmbigdata1

192.168.86.31  ubuntucmbigdata1
192.168.86.32 ubuntucmbigdata2
192.168.86.33 ubuntucmbigdata3
192.168.86.34 ubuntucmbigdata4

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

  主机名

bigdata@ubuntucmbigdata1:~$ sudo vim /etc/hostname
bigdata@ubuntucmbigdata2:~$ sudo vim /etc/hostname
bigdata@ubuntucmbigdata3:~$ sudo vim /etc/hostname
bigdata@ubuntucmbigdata4:~$ sudo vim /etc/hostname

  这里肯定是对的。

配置ssh免密码登录 (对于这一步,其实可以不做,因为,在CM时会自动安装,大家可以跳过)

  见

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5)(一)

bigdata@ubuntucmbigdata1:~$ mkdir .ssh
bigdata@ubuntucmbigdata1:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/bigdata/.ssh/id_rsa):(回车)
Enter passphrase (empty for no passphrase):(回车)
Enter same passphrase again: (回车)
Your identification has been saved in /home/bigdata/.ssh/id_rsa.
Your public key has been saved in /home/bigdata/.ssh/id_rsa.pub.
The key fingerprint is:
58:c2:3f:25:cb:82:5b:d5:b4:b2:ea:54:ac:e2:ea:c8 bigdata@ubuntucmbigdata1
The key's randomart image is:
+--[ RSA 2048]----+
| . |
| . o . |
| o = + |
| . O * |
| . + S |
| o = . |
| o + |
|.. . + |
|.Eo.. . |
+-----------------+
bigdata@ubuntucmbigdata1:~$ cd .ssh
bigdata@ubuntucmbigdata1:~/.ssh$ cat id_rsa.pub >> authorized_keys
bigdata@ubuntucmbigdata1:~/.ssh$ cd ..
bigdata@ubuntucmbigdata1:~$ chmod 700 .ssh
bigdata@ubuntucmbigdata1:~$ chmod 600 .ssh/*
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
The authenticity of host 'ubuntucmbigdata1 (127.0.1.1)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 14:45:01 2017 from 192.168.86.1
bigdata@ubuntucmbigdata1:~$ exit;
logout
Connection to ubuntucmbigdata1 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:52:15 2017 from localhost
bigdata@ubuntucmbigdata1:~$ exit;

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

  将所有节点(ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4)中的共钥id_ras.pub拷贝到ubuntucmbigdata1中的authorized_keys文件中。

bigdata@ubuntucmbigdata2:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password:
bigdata@ubuntucmbigdata2:~$

bigdata@ubuntucmbigdata3:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password:
bigdata@ubuntucmbigdata3:~$

bigdata@ubuntucmbigdata4:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password:
bigdata@ubuntucmbigdata4:~$

  

  得到

bigdata@ubuntucmbigdata1:~$ cd .ssh
bigdata@ubuntucmbigdata1:~/.ssh$ ls
authorized_keys id_rsa id_rsa.pub known_hosts
bigdata@ubuntucmbigdata1:~/.ssh$ cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrEPrX5HHQCnwWoavXUoM8N+fINUckzZ7EcDjJqs1EYkrzltdmMrKQv9VeGGj2BPaGoxzmgit+DPFGuPmPgftBOPWQnLVoILqtQd+H5m0wrK67W7Y98cUpZ+0thCbnmUZRuAES1mt2xS/bL2ibw8AvRrd6HUHOhH17Tcd/2gok32ximn48kPzs6li5d2ay8rPqgqD3QQw3NlAhVmK7RVmpsvlHCTnhrQIoNCz8A+adtK5r1jSgH7uKmPAJTqMUP2Zgv02XQTf/kjW3eacNAWR96Khdh0HsktaK6HuC+FFC+8bXqAj4Hih25EX8RTMFMl3nhGT5FPMLyt7C0Z65QhKV bigdata@ubuntucmbigdata1
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDi7mZhUMzB1h7XjWPIZo3w/Tps0Oxr3mgOeKVfoD4pme/lzgTz61FpEmgggc2WB+mNLKSG/WiazTzuSXKUdBXnM5LNeCGV3ZcWBxFrwp2+wAYH6L5wOGqRq+oQzKUHIsvqEc7TBpN3JmaqZOw4zVCwRBBzQ4cg1B2SjSsjDFLBpjlfM+N8UDG+g9OaZDlEu1z5oUyz0KsKbBtvkkSott+sB+EX4zYSDz/cwUPmSf76RJLk89AI8aX4wGS+P4Eahs5/J2wfLOnAWBG2NQoYVPKyN4sXj7OWw+aErriUJXOqLayYV5xSM2yJjT9WuzbNOiR/D7HGZBlC4wXWG+11GdO5 bigdata@ubuntucmbigdata2
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZtE7Z4cXPjv/t0bC4Ozi3HuovI6XiNe+tJ7cQ6WKp/m44a8w/FqgB4HVZLLQusuXwQM/mEzHKiL1EdPsi5om/s7A7xyDakl2OSUOqki871+/TSjFVTUOOz/H/wwHvl2QzBkb8/dMZLphU1/E4odrFhhlhGGpjK5WUdY8cYH6yyli6W3oDPZ55UG+8EnM89T1AyhC1E81xmyMzGMCj4603oZFwckOeff5n2dqeXg0CRhXO9HhsF9kSZQnE2GmR5pLZlORMkitsCKIhaf5UOBzrGENeJylHGCV1F+KlYjQ0ND4ufTUq14aAnUwBOCqCMhB1tBOBGI4O3S0+j27xumwN bigdata@ubuntucmbigdata3
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXPtqzCU+AmRi+l5lysCEG8gbFKPItYjczvSZIe6+arOfgE/3Ozu2cwr9MABTjgd80hk7ZG4YNRxBPxKvXKtcZxGAe6poe1G5UOrlKOMLeqv1mnUiwtRVsiWp0qnPFcmMt17mpCyN7XnCOLTsdTcl+BVl0c2S2i1xoC6vRKvqV1C/nneJUmuD3KRR1zmR3+a9Wj4yhRRahO6ro/LhgqdwrEaR+NOuT+/1d058fXANToaqyUMa72pz5M2kdxZprumzmHAJ5KlOYImlGrahdLk1wPFckA2m1BMKKycxZHFm1X1O9HeSFrD9N+pdtZY1CXdc6N9hmWKzjBDqw71rZgdUd bigdata@ubuntucmbigdata4
bigdata@ubuntucmbigdata1:~/.ssh$

  说明成功!

  然后将中ubuntucmbigdata1的authorized_keys文件分发到所有节点上面。

bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata2:~/.ssh/
The authenticity of host 'ubuntucmbigdata2 (192.168.86.32)' can't be established.
ECDSA key fingerprint is 17:a2:fb:1e:47:dc:79:d5:2d:5a:aa:79:9b:10:c2:eb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata2,192.168.86.32' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata2's password:
authorized_keys 100% 1624 1.6KB/s 00:00
bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata3:~/.ssh/
The authenticity of host 'ubuntucmbigdata3 (192.168.86.33)' can't be established.
ECDSA key fingerprint is 6a:32:65:61:98:78:3f:ad:08:2c:f3:a1:bc:c6:c4:91.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata3,192.168.86.33' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata3's password:
authorized_keys 100% 1624 1.6KB/s 00:00
bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata4:~/.ssh/
The authenticity of host 'ubuntucmbigdata4 (192.168.86.34)' can't be established.
ECDSA key fingerprint is f4:d5:92:17:dc:b1:8f:95:61:11:95:cd:d3:fa:2e:57.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata4,192.168.86.34' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata4's password:
authorized_keys 100% 1624 1.6KB/s 00:00
bigdata@ubuntucmbigdata1:~/.ssh$

  大家通过ssh 相互访问,如果都能无密码访问,代表ssh配置成功。这里很简单,我就不多赘述了。

bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:52:23 2017 from localhost
bigdata@ubuntucmbigdata1:~$ exit;Connection to ubuntucmbigdata1 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata2
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:56:00 2017 from localhost
bigdata@ubuntucmbigdata2:~$ exit;
logout
Connection to ubuntucmbigdata2 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata3
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:56:14 2017 from localhost
bigdata@ubuntucmbigdata3:~$ exit;
logout
Connection to ubuntucmbigdata3 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata4
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:56:27 2017 from localhost
bigdata@ubuntucmbigdata4:~$ exit;
logout
Connection to ubuntucmbigdata4 closed.
bigdata@ubuntucmbigdata1:~$

  这样,就说明是成功!

  

JDK的安装(建议1.8)(对于这一步,其实可以不做,因为,在CM时会自动安装,大家可以跳过)(但是要先卸载linux系统自带的openjdk,因为自动也是安装的是orcal公司的)

  见

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5)(一)

  这里,因为是ubuntu14.04版本,作为专业人员来说,

Ubuntu14.04下Neo4j图数据库官网安装部署步骤(图文详解)(博主推荐)

bigdata@ubuntucmbigdata1:~$ sudo apt-get remove openjdk* 

bigdata@ubuntucmbigdata1:/usr/local$ pwd
/usr/local
bigdata@ubuntucmbigdata1:/usr/local$ ll
total 40
drwxr-xr-x 10 root root 4096 2月 18 2016 ./
drwxr-xr-x 10 root root 4096 2月 18 2016 ../
drwxr-xr-x 2 root root 4096 2月 18 2016 bin/
drwxr-xr-x 2 root root 4096 2月 18 2016 etc/
drwxr-xr-x 2 root root 4096 2月 18 2016 games/
drwxr-xr-x 2 root root 4096 2月 18 2016 include/
drwxr-xr-x 4 root root 4096 2月 18 2016 lib/
lrwxrwxrwx 1 root root 9 6月 24 15:00 man -> share/man/
drwxr-xr-x 2 root root 4096 2月 18 2016 sbin/
drwxr-xr-x 7 root root 4096 2月 18 2016 share/
drwxr-xr-x 2 root root 4096 2月 18 2016 src/
bigdata@ubuntucmbigdata1:/usr/local$ sudo mkdir jdk
bigdata@ubuntucmbigdata1:/usr/local$ ll
total 44
drwxr-xr-x 11 root root 4096 6月 25 16:08 ./
drwxr-xr-x 10 root root 4096 2月 18 2016 ../
drwxr-xr-x 2 root root 4096 2月 18 2016 bin/
drwxr-xr-x 2 root root 4096 2月 18 2016 etc/
drwxr-xr-x 2 root root 4096 2月 18 2016 games/
drwxr-xr-x 2 root root 4096 2月 18 2016 include/
drwxr-xr-x 2 root root 4096 6月 25 16:08 jdk/
drwxr-xr-x 4 root root 4096 2月 18 2016 lib/
lrwxrwxrwx 1 root root 9 6月 24 15:00 man -> share/man/
drwxr-xr-x 2 root root 4096 2月 18 2016 sbin/
drwxr-xr-x 7 root root 4096 2月 18 2016 share/
drwxr-xr-x 2 root root 4096 2月 18 2016 src/
bigdata@ubuntucmbigdata1:/usr/local$ sudo chown -R bigdata:bigdata jdk/
bigdata@ubuntucmbigdata1:/usr/local$ cd jdk/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ pwd
/usr/local/jdk
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 177000
drwxr-xr-x 2 bigdata bigdata 4096 6月 25 16:09 ./
drwxr-xr-x 11 root root 4096 6月 25 16:08 ../
-rw-r--r-- 1 root root 181238643 9月 3 2016 jdk-8u60-linux-x64.tar.gz
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo tar -zxvf jdk-8u60-linux-x64.tar.gz

bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 177004
drwxr-xr-x 3 bigdata bigdata 4096 6月 25 16:09 ./
drwxr-xr-x 11 root root 4096 6月 25 16:08 ../
drwxr-xr-x 8 uucp 143 4096 8月 5 2015 jdk1.8.0_60/
-rw-r--r-- 1 root root 181238643 9月 3 2016 jdk-8u60-linux-x64.tar.gz
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo rm jdk-8u60-linux-x64.tar.gz
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo chown -R bigdata:bigdata jdk1.8.0_60/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 12
drwxr-xr-x 3 bigdata bigdata 4096 6月 25 16:10 ./
drwxr-xr-x 11 root root 4096 6月 25 16:08 ../
drwxr-xr-x 8 bigdata bigdata 4096 8月 5 2015 jdk1.8.0_60/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo vim /etc/profile
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo source /etc/profile
sudo: source: command not found
bigdata@ubuntucmbigdata1:/usr/local/jdk$ source /etc/profile
bigdata@ubuntucmbigdata1:/usr/local/jdk$ java -version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
bigdata@ubuntucmbigdata1:/usr/local/jdk$

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

以及,我们还可以设置文件打开数量和用于最大进程数

  文件打开数量

  参考

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5)(一)

bigdata@ubuntucmbigdata1:~$ sudo vi /etc/security/limits.conf 

* soft nofile 65535
* hard nofile 65535
* soft nproc 32000
* hard nproc 32000

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

   ubuntu系统的root用户开启

  参考

Ubuntu14.04安装之后的一些配置

bigdata@ubuntucmbigdata1:~$ sudo -s
root@ubuntucmbigdata1:~# pwd
/home/bigdata
root@ubuntucmbigdata1:~#

 MySQL的安装(以下步骤仅仅在ubuntucmbigdata1这台机器上做)

  注意:Cloudera-Manages-Server 端 需要安装 MySQL,Cloudera-Manages-Agent 端 可以不装。

  即,在我这里就是,Cloudera-Manages-Server 是cmbigdata1。同时,我也是喜欢把oozie、hive等这些,都是安装在cmbigdata1这台机器上。

  Cloudera-Manages-Agent,在我这里就是,cmbigdata2、cmbigdata3和cmbigdata4。

  参考

60分钟内从零起步驾驭Hive实战学习笔记(Ubuntu里安装mysql)

Ubuntu 14.04 MySQL 安装配置 For Cloudera Manages 5

root@ubuntucmbigdata1:~# apt-get install mysql-server

。。。
。。。
Setting up libdbd-mysql-perl (4.025-1ubuntu0.1) ...
Setting up libterm-readkey-perl (2.31-1) ...
Setting up mysql-client-5.5 (5.5.55-0ubuntu0.14.04.1) ...
Setting up mysql-server-5.5 (5.5.55-0ubuntu0.14.04.1) ...
170625 16:28:33 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
170625 16:28:33 [Note] Ignoring --secure-file-priv value as server is running with --bootstrap.
170625 16:28:33 [Note] /usr/sbin/mysqld (mysqld 5.5.55-0ubuntu0.14.04.1) starting as process 7930 ...
mysql start/running, process 8062
Setting up libhtml-template-perl (2.95-1) ...
Processing triggers for ureadahead (0.100.0-16) ...
Setting up mysql-server (5.5.55-0ubuntu0.14.04.1) ...
Processing triggers for libc-bin (2.19-0ubuntu6.7) ...
root@ubuntucmbigdata1:~#

安装成功!

  启动mysql服务

安装MySql成功后,mysql服务会自动启动

root@ubuntucmbigdata1:~# service mysql start

或者
root@ubuntucmbigdata1:~# sudo /etc/init.d/mysql start

  安装mysql JDBC驱动

root@ubuntucmbigdata1:~# apt-get install libmysql-java

设置MySQL的root用户设置密码,这一步,就不需要做了,在CentOS版本里,需要做。在Ubuntu版本,在安装时,就已经设置了。

  跳过,就是

创建Cloudera-Manages-Server 需要用到的数据库表 for MySql。

  Creating Databases for Activity Monitor, Reports Manager, Hive Metastore Server, Sentry Server, Cloudera Navigator AuditServer, and Cloudera Navigator Metadata Server

root@ubuntucmbigdata1:~# mysql -root -p
Enter password: (密码为rootroot)
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 42
Server version: 5.5.55-0ubuntu0.14.04.1 (Ubuntu) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>

  执行创建数据库脚本语句。

mysql> create database hive DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> grant all on *.* TO 'hive'@'ubuntucmbigdata1' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> create database oozie DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> grant all on *.* TO 'oozie'@'ubuntucmbigdata1' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> grant all on oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> create database amon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on amon.* TO 'amon'@'localhost' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'amon'@'ubuntucmbigdata1' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> create database smon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on smon.* TO 'smon'@'localhost' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'smon'@'ubuntucmbigdata1' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> create database rman DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on rman.* TO 'rman'@'localhost' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> grant all on *.* TO 'rman'@'ubuntucmbigdata1' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> create database hmon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on hmon.* TO 'hmon'@'localhost' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'hmon'@'ubuntucmbigdata1' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> create database nav DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on nav.* TO 'nav'@'localhost' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> grant all on *.* TO 'nav'@'ubuntucmbigdata1' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> grant all on nav.* TO 'nav'@'%' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql>

mysql> flush privileges;
Query OK, rows affected (0.08 sec) mysql> select user,host,password from mysql.user;
+------------------+------------------+-------------------------------------------+
| user | host | password |
+------------------+------------------+-------------------------------------------+
| root | localhost | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root | ubuntucmbigdata1 | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root | 127.0.0.1 | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root | :: | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| debian-sys-maint | localhost | *0C134AA3949C6DEE1804B1D1D641A7C2824C599A |
| scm | % | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| scm | ubuntucmbigdata1 | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| scm | localhost | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| hive | localhost | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| hive | ubuntucmbigdata1 | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| hive | % | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| oozie | localhost | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| oozie | ubuntucmbigdata1 | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| oozie | % | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| amon | localhost | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| amon | ubuntucmbigdata1 | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| amon | % | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| smon | localhost | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| smon | ubuntucmbigdata1 | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| smon | % | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| rman | localhost | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| rman | ubuntucmbigdata1 | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| rman | % | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| hmon | localhost | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| hmon | ubuntucmbigdata1 | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| hmon | % | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| nav | localhost | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
| nav | ubuntucmbigdata1 | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
| nav | % | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
+------------------+------------------+-------------------------------------------+
rows in set (0.00 sec) mysql> exit;
欢迎大家,加入我的微信公众号:大数据躺过的坑     免费给分享
 
 
 

同时,大家可以关注我的个人博客

 http://www.cnblogs.com/zlslch/   和  http://www.cnblogs.com/lchzls/ 

  人生苦短,我愿分享。本公众号将秉持活到老学到老学习无休止的交流分享开源精神,汇聚于互联网和个人学习工作的精华干货知识,一切来于互联网,反馈回互联网。
  目前研究领域:大数据、机器学习、深度学习、人工智能、数据挖掘、数据分析。 语言涉及:Java、Scala、Python、Shell、Linux等 。同时还涉及平常所使用的手机、电脑和互联网上的使用技巧、问题和实用软件。 只要你一直关注和呆在群里,每天必须有收获

以及对应本平台的QQ群:161156071(大数据躺过的坑)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)的更多相关文章

  1. Cloudera Manager安装之利用parcels方式(在线或离线)安装3或4节点集群(包含最新稳定版本或指定版本的安装)(添加服务)(Ubuntu14.04)(五)

    前期博客 Cloudera Manager安装之Cloudera Manager 5.6.X安装(tar方式.rpm方式和yum方式) (Ubuntu14.04) (三) 如果大家,在启动的时候,比如 ...

  2. Cloudera Manager安装之利用parcels方式(在线或离线)安装单节点集群(包含最新稳定版本或指定版本的安装)(添加服务)(Ubuntu14.04)(四)

    .. 欢迎大家,加入我的微信公众号:大数据躺过的坑     免费给分享       同时,大家可以关注我的个人博客:  http://www.cnblogs.com/zlslch/   和  http ...

  3. Ubuntu14.04下Cloudera安装搭建部署大数据集群(图文分五大步详解)(博主强烈推荐)(在线或离线)

    第一步: Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一) 第二步: Cloudera Manager安装之时间服务器和时间客户端(Ub ...

  4. Ubuntu14.04下完美安装cloudermanage多种方式(图文详解)(博主推荐)

    说在前面的话 我的机器是总共4台,分别为ubuntucmbigdata1.ubuntucmbigdata2.ubuntucmbigdata3和ubuntucmbigdata4. ClouderaMan ...

  5. Ubuntu14.04下Mongodb(在线安装方式|apt-get)安装部署步骤(图文详解)(博主推荐)

    不多说,直接上干货! 本博文介绍了MongoDB,并详细指引读者在Ubuntu下MongoDB的安装和使用.本教程在Ubuntu14.04下测试通过. 一.MongoDB介绍 MongoDB 是一个是 ...

  6. Cloudera Manager 5和CDH5离线安装

    CDH (Cloudera’s Distribution, including Apache Hadoop),是Hadoop众多分支中的一种,由Cloudera维护,基于稳定版本的Apache Had ...

  7. Cloudera Manager和CDH5.8离线安装

    https://blog.csdn.net/zzq900503/article/details/52982828 简介 我们在上篇文章中已经了解了CDH,为了后续的学习,我们本章就来安装CDH5.8. ...

  8. [转]Cloudera Manager和CDH5.8离线安装

    https://blog.csdn.net/zzq900503/article/details/52982828 https://www.cnblogs.com/felixzh/p/9082344.h ...

  9. hadoop基础----hadoop实战(七)-----hadoop管理工具---使用Cloudera Manager安装Hadoop---Cloudera Manager和CDH5.8离线安装

    hadoop基础----hadoop实战(六)-----hadoop管理工具---Cloudera Manager---CDH介绍 简介 我们在上篇文章中已经了解了CDH,为了后续的学习,我们本章就来 ...

随机推荐

  1. 对ArrayList(Vector)的排序

    当需要对ArrayList(Vector)里面的元素进行简单的排序时,可以使用Collections.sort();这个方法 import java.util.ArrayList; import ja ...

  2. CentOS下安装PHP的AMQP扩展方法和步骤

    AMQP,即Advanced Message Queuing Protocol,一个提供统一消息服务的应用层标准高级消息队列协议,是应用层协议的一个开放标准,为面向消息的中间件设计.基于此协议的客户端 ...

  3. RobotFramework做接口自动化(post请求)

    接口成功时返回: { "reCorde": "SUCCESS", "data": { ", "verify": ...

  4. django view 装饰器

    Django提供了几个可以应用于视图以支持各种HTTP特性的装饰器 Allowed HTTP django.views.decorators.http里的装饰器可以根据请求方法限制对视图的访问. re ...

  5. PageAdmin环境配置要求

    1.操作系统要求: Win7/win8/win2008/win2012及以上版本都可以,建议用64位的操作系统,服务器建议选择win2012或以上版本. 2.net framework版本要求: ne ...

  6. react-dnd使用介绍

    核心API 想要灵活使用,就先知道几个核心API DragSource 用于包装你需要拖动的组件,使组件能够被拖拽(make it draggable) DropTarget 用于包装接收拖拽元素的组 ...

  7. Blocks to Cubes

    Bholu the Pandit on this New Year wanted to divide his Cuboidal Packaging block into cubes. But he l ...

  8. iOS 图片的存储以及读取和删除

    将图片存储到本地 NSArray *dirArray = NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask ...

  9. Setup "EQGRP_Lost_in_Translation" Of NAS

    Setup "EQGRP_Lost_in_Translation" Of NAS 1.前言: 北京时间4月14号晚,TheShadowBrokers在steemit.com博客上放 ...

  10. nginx负载均衡配合keepalived服务案例实战

    本实验用4台 centos6 虚拟机,2台做负载均衡,2台做web服务器,都先装上nginx lb01:192.168.0.235  --主负载均衡器 lb02:192.168.0.236  --备负 ...