ubuntu14.04离线配置cm5.5.1

cd /opt/cm-5.5.1/具体参考http://www.aboutyun.com/thread-10852-1-1.html

离线安装：下载所需文件

http://archive.cloudera.com/cm5/cm/5/中的cloudera-manager-trusty-cm5.5.1_amd64.tar.gz

http://archive.cloudera.com/cdh5/parcels/5.5.1/ 中的CDH-5.5.1-1.cdh5.5.1.p0.11-trusty.parcel和 CDH-5.5.1-1.cdh5.5.1.p0.11-trusty.parcel.sha1，还有manifest.json

mysql配置

http://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5_5

所有机器：

apt-get install ntp

修改主机名/etc/hostname

修改/etc/hosts只保留127.0.0.1 localhost

dpkg -i oracle-j2sdk1.7_1.7.0+update67-1_amd64.deb

CM Server节点配置mysql数据库

apt-get install mysql-server

cm server:

创建数据库：

#hive
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

#activity monitor
create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;

tar zxvf /cloudera-manager-trusty-cm5.5.1_amd64.tar.gz

然后mv cm-5.5.1/ /opt

将mysql-connector添加到/opt/cm-5.5.1/share/cmf/lib/

初始化mysql的库

cd /opt/cm-5.5.1/

share/cmf/schema/scm_prepare_database.sh mysql cm -hlocalhost -uroot -proot --scm-host localhost scm scm scm

修改/opt/cm-5.5.1/etc/cloudera-scm-agent/config.ini中的server_host为主节点的主机名

将/opt/cm-5.5.1同步到所有节点

所有节点创建用户

useradd --system --home-dir=/opt/cm-5.5.1/run/cloudera-scm-server --no-create-home --shell=/bin/false  cloudera-scm

将CHD5相关的Parcel包放到主节点的/opt/cloudera/parcel-repo/(自己创建)目录中

CDH-5.5.1-1.cdh5.5.1.p0.11-trusty.parcel

CDH-5.5.1-1.cdh5.5.1.p0.11-trusty.parcel.sha

启动server

/opt/cm-5.5.1/etc/init.d/cloudera-scm-server start

启动所有agent

/opt/cm-5.5.1/etc/init.d/cloudera-scm-agent start

cp /mysql-connector-java-5.1.38-bin.jar  /usr/share/java/mysql-connector-java.jar

cp /mysql-connector-java-5.1.38-bin.jar  /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hive/lib/

访问：http://server:7180

关闭虚拟内存：echo 0 > /proc/sys/vm/swappiness

create database oozie;

错误：

Traceback (most recent call last):
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/bin/hue", line 12, in <module>
    load_entry_point('desktop==3.9.0', 'console_scripts', 'hue')()
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/desktop/core/src/desktop/manage_entry.py", line 57, in entry
    execute_from_command_line(sys.argv)
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/management/base.py", line 284, in execute
    self.validate()
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/management/base.py", line 310, in validate
    num_errors = get_validation_errors(s, app)
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/core/management/validation.py", line 34, in get_validation_errors
    for (app_name, error) in get_app_errors().items():
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/loading.py", line 196, in get_app_errors
    self._populate()
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/loading.py", line 78, in _populate
    self.load_app(app_name)
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/db/models/loading.py", line 99, in load_app
    models = import_module('%s.models' % app_name)
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/Django-1.6.10-py2.7.egg/django/utils/importlib.py", line 40, in import_module
    __import__(name)
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/apps/jobbrowser/src/jobbrowser/models.py", line 20, in <module>
    import lxml.html
  File "/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hue/build/env/lib/python2.7/site-packages/lxml-2.2.2-py2.7-linux-x86_64.egg/lxml/html/__init__.py", line 12, in <module>
    from lxml import etree
ImportError: libxslt.so.1: cannot open shared object file: No such file or directory

解决方法：apt-get install python-libxslt1

一些简单操作：

sudo -u hdfs hadoop fs -put /test.txt  /hh/

运行wordcount

sudo -u hdfs hadoop jar /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep /hh /out 'd[a-z.]+'

查看结果：

sudo -u hdfs hadoop fs -cat /out/*
sudo -u hdfs hadoop fs -rmr /out

关闭节点服务：

/opt/cm-5.5.1/etc/init.d/cloudera-scm-agent hard_stop_confirmed