hue安装与部署

运行环境

• centOS 6.6
• hadoop 2.4.0
• hive 1.2.0
• spark 1.4.1
• HUE 3.9

介绍:

Hue是一个开源的Apache Hadoop UI系统，最早是由Cloudera Desktop演化而来，由Cloudera贡献给开源社区，它是基于Python Web框架Django实现的。通过使用Hue我们可以在浏览器端的Web控制台上与Hadoop集群进行交互来分析处理数据，例如操作HDFS上的数据，运行MapReduce Job等等。

HUE功能:

• 数据库查询编辑器,支持 Hive, Impala, MySql, PostGres, Sqlite and Oracle
• 动态查询仪表盘,支持 Solr
• 支持 Spark 编辑器和仪表盘
• 浏览器查看状态,支持 YARN, HDFS, Hive table Metastore, HBase, ZooKeeper
• 支持 Pig Editor, Sqoop2, Oozie workflows 编辑器和仪表盘
• 将数据导入hdfs

在centOS编译需要的依赖

• ant
• asciidoc
• cyrus-sasl-devel
• cyrus-sasl-gssapi
• gcc
• gcc-c++
• krb5-devel
• libtidy (for unit tests only)
• libxml2-devel
• libxslt-devel
• make
• mvn (from maven package or maven3 tarball)
• mysql
• mysql-devel
• openldap-devel
• python-devel
• sqlite-devel
• openssl-devel (for version 7+)
• gmp-devel

添加maven源

sudo wget http://repos.fedorapeople.org/repos/dchen/apache-maven/epel-apache-maven.repo -O /etc/yum.repos.d/epel-apache-maven.repo

安装依赖

sudo yum install apache-maven ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel

源码下载

下载，编译源码(hue 3.9),编译时间较长

git clone https://github.com/cloudera/hue.git branch-3.9
cd branch-3.9
make apps

编译完后也可以选择安装

make install

在desktop/conf/pseudo-distributed.ini中修改配置

假设所有服务都运行在testHost下

[desktop]
   # hue webServer 地址和端口号
   http_host=192.168.47.213
   http_port=8001

  # Webserver runs as this user
   server_user=hue
   server_group=hue

  # This should be the Hue admin and proxy user
   default_user=hue

  # This should be the hadoop cluster admin
   default_hdfs_superuser=admin

[hadoop]
    fs_defaultfs=tdhdfs
    webhdfs_url=http://192.168.47.216:50070/webhdfs/v1/
    hadoop_conf_dir=/usr/install/hadoop/etc/hadoop/conf

[[yarn_clusters]]
      # Enter the host on which you are running the ResourceManager
      resourcemanager_host=192.168.47.216

      # The port where the ResourceManager IPC listens on
      resourcemanager_port=23140

      # Whether to submit jobs to this cluster
      submit_to=True

      # URL of the ResourceManager API
      resourcemanager_api_url=http://192.168.47.216:23188

      # URL of the ProxyServer API
      proxy_api_url=http://192.168.47.216:8888

      # URL of the HistoryServer API
      history_server_api_url=http://192.168.47.216:19888

[beeswax]
  # If Kerberos security is enabled, use fully-qualified domain name (FQDN).
  hive_server_host=192.168.47.213

  # Port where HiveServer2 Thrift server runs on.
   hive_server_port=10000

  # Hive configuration directory, where hive-site.xml is located
   hive_conf_dir=/usr/install/apache-hive-1.2.0-bin/conf

[zookeeper]
host_ports=zk1:2181,zk2:2181,zk3:2181

启动

build/env/bin/supervisor

运行界面

一些问题

使用hive 0.13.0时会报

can't fetch result之类的错误

是版本原因，升级hive版本为1.2.0问题消失

连接hive error

Could not install table: Error creating table sample_07: Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found (code THRIFTTRANSPORT): TTransportException('Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found',).

解决方案：

yum -y install cyrus-sasl-plain cyrus-sasl-devel cyrus-sasl-gssapi