hue集成各种组件

一、Hue安装

可以编译安装，我这里有已经编译好的，直接解压使用；

hue默认端口：8888

http://gethue.com/

https://github.com/cloudera/hue

http://archive.cloudera.com/cdh5/cdh/5/hue-3.7.0-cdh5.3.6/manual.html#_install_hue

1、先停止oozie 、hdfs服务

##oozie
[root@hadoop-senior oozie-4.0.0-cdh5.3.6]# bin/oozied.sh stop

##hdfs

[root@hadoop-senior hadoop-2.5.0-cdh5.3.6]# sbin/hadoop-daemon.sh stop namenode



[root@hadoop-senior hadoop-2.5.0-cdh5.3.6]# sbin/hadoop-daemon.sh stop datanode

2、安装配置hue

##
[root@hadoop-senior ~]# mkdir /opt/app

##
[root@hadoop-senior hue]# tar zxf hue-3.7.0-cdh5.3.6-build.tar.gz -C /opt/app/

[root@hadoop-senior hue]# cd /opt/app/hue-3.7.0-cdh5.3.6/
[root@hadoop-senior hue-3.7.0-cdh5.3.6]# ls
app.reg  apps  build  cloudera  desktop  docs  ext  LICENSE.txt  Makefile  Makefile.sdk  Makefile.vars  Makefile.vars.priv  maven  NOTICE.txt  README  tools  VERSION

##配置：/opt/app/hue-3.7.0-cdh5.3.6/desktop/conf/hue.ini   如下：
  secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o

  # Webserver listens on this address and port
  http_host=hadoop-senior.ibeifeng.com
  http_port=8888

  # Time zone name
  time_zone=Asia/Shanghai

  # Enable or disable Django debug mode.
  django_debug_mode=false

  # Enable or disable backtrace for server error
  http_500_debug_mode=false

二、hue集成hdfs、yarn

1、配置hdfs

##hdfs-site.xml
<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>

##core-site.xml
<property>
  <name>hadoop.proxyuser.hue.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.hue.groups</name>
  <value>*</value>
</property>

##启动hdfs
[root@hadoop-senior hadoop-2.5.0-cdh5.3.6]# sbin/hadoop-daemon.sh start namenode
[root@hadoop-senior hadoop-2.5.0-cdh5.3.6]# sbin/hadoop-daemon.sh start datanode

##配置hue
/opt/app/hue-3.7.0-cdh5.3.6/desktop/conf/hue.ini      如下，大概在580行以后：
#
fs_defaultfs=hdfs://hadoop-senior.ibeifeng.com:8020
#
webhdfs_url=http://hadoop-senior.ibeifeng.com:50070/webhdfs/v1
#
hadoop_conf_dir=/opt/cdh-5.3.6/hadoop-2.5.0-cdh5.3.6/etc/hadoop

2、配置yarn

##/opt/app/hue-3.7.0-cdh5.3.6/desktop/conf/hue.ini 如下：
resourcemanager_host=hadoop-senior.ibeifeng.com
resourcemanager_port=8032
submit_to=True
resourcemanager_api_url=http://hadoop-senior.ibeifeng.com:8088
proxy_api_url=http://hadoop-senior.ibeifeng.com:8088
history_server_api_url=http://hadoop-senior.ibeifeng.com:19888

##启动hue
因为我之前都是用root用户操作的，而hue不让用root启动；
我这里新建一个用户，来启动hue：
[root@hadoop-senior hue-3.7.0-cdh5.3.6]# useradd beifeng
[root@hadoop-senior hue-3.7.0-cdh5.3.6]# passwd beifeng
[root@hadoop-senior hue-3.7.0-cdh5.3.6]# chown -R beifeng /opt/app/hue-3.7.0-cdh5.3.6/
[beifeng@hadoop-senior hue-3.7.0-cdh5.3.6]$ ./build/env/bin/supervisor     //启动hue

然后在浏览器中ip:端口 即可访问；

三、Hue集成Hive

1、配置hive server2

##hue集成hive 依赖于hive server2

##配置hive  ，hive-site.xml
    <property>
        <name>hive.server2.thrift.port</name>
        <value>10000</value>
    </property>

    <property>
        <name>hive.server2.thrift.bind.host</name>
        <value>hadoop-senior.ibeifeng.com</value>
    </property>

##启动hive server2
[root@hadoop-senior ~]# /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/bin/hiveserver2        //启动在前端

2、hive metastore

##hive-site.xml
    <property>
        <name>hive.metastore.uris</name>
        <value>thrift://hadoop-senior.ibeifeng.com:9083</value>
    </property>

##启动metastore
[root@hadoop-senior ~]# /opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/bin/hive --service metastore

3、配置hue

##hue.ini    //大概在720多行
hive_server_host=hadoop-senior.ibeifeng.com
hive_server_port=10000
hive_conf_dir=/opt/cdh-5.3.6/hive-0.13.1-cdh5.3.6/conf
server_conn_timeout=120

##重启hue

四、hue集成RDBMS

1、hue.ini

##大概在510行，这里需要配置sqlite，hue本身使用的是sqlite，然后再配置集成mysql
  [[databases]]
    # sqlite configuration.
    [[[sqlite]]]
      # Name to show in the UI.
      nice_name=SQLite

      # For SQLite, name defines the path to the database.
      name=/opt/app/hue-3.7.0-cdh5.3.6/desktop/desktop.db

      # Database backend to use.
      engine=sqlite

      # Database options to send to the server when connecting.
      # https://docs.djangoproject.com/en/1.4/ref/databases/
      ## options={}

    # mysql, oracle, or postgresql configuration.
    [[[mysql]]]
      # Name to show in the UI.
      nice_name="My SQL DB"

      # For MySQL and PostgreSQL, name is the name of the database.
      # For Oracle, Name is instance of the Oracle server. For express edition
      # this is 'xe' by default.
      name=test

      # Database backend to use. This can be:
      # 1. mysql
      # 2. postgresql
      # 3. oracle
      engine=mysql

      # IP or hostname of the database to connect to.
      host=hadoop-senior.ibeifeng.com

      # Port the database server is listening to. Defaults are:
      # 1. MySQL: 3306
      # 2. PostgreSQL: 5432
      # 3. Oracle Express Edition: 1521
      port=3306

      # Username to authenticate with when connecting to the database.
      user=root

      # Password matching the username to authenticate with when
      # connecting to the database.
      password=123456

      # Database options to send to the server when connecting.
      # https://docs.djangoproject.com/en/1.4/ref/databases/
      ## options={}

##重启hue

五、hue集成oozie

1、hue.ini

local_data_dir=/opt/cdh-5.3.6/oozie-4.0.0-cdh5.3.6/examples

sample_data_dir=/opt/cdh-5.3.6/oozie-4.0.0-cdh5.3.6/examples/input-data

remote_data_dir=/user/root/examples/apps

oozie_jobs_count=100

2、oozie-site.xml

配置/user/oozie/share/lib

<property>
        <name>oozie.service.WorkflowAppService.system.libpath</name>
        <value>/user/oozie/share/lib</value>
        <description>
            System library path to use for workflow applications.
            This path is added to workflow application if their job properties sets
            the property 'oozie.use.system.libpath' to true.
        </description>
    </property>

3、执行下列命令

##在oozie中执行
bin/oozie-setup.sh sharelib create \
-fs hdfs://hadoop-senior.ibeifeng.com:8020 \
-locallib oozie-sharelib-4.0.0-cdh5.3.6-yarn.tar.gz

##重启oozie
[root@hadoop-senior oozie-4.0.0-cdh5.3.6]# bin/oozied.sh stop
[root@hadoop-senior oozie-4.0.0-cdh5.3.6]# bin/oozied.sh start


##重启hue