HBase安装与应用

1.  参考说明

参考文档：

https://hbase.apache.org/book.html

2.  安装环境说明

2.1.  环境说明

CentOS7.4+ Hadoop2.7.5的伪分布式环境

主机名	NameNode	SecondaryNameNode	DataNodes
centoshadoop.smartmap.com	192.168.1.80	192.168.1.80	192.168.1.80

Hadoop的安装目录为：/opt/hadoop/hadoop-2.7.5

3.  安装

3.1.  Zookeeper

3.1.1.  Zookeeper下载

http://zookeeper.apache.org/releases.html#download

3.1.2.  Zookeeper解压

将下载的zookeeper-3.4.12.tar.gz解压到/opt/hadoop/zookeeper-3.4.12目录下

3.1.3.  Zookeeper配置

3.1.3.1. 创建目录

[root@centoshadoop conf]# mkdir -p /opt/hadoop/zookeeper-3.4.12/data/db

[root@centoshadoop conf]# mkdir -p
/opt/hadoop/zookeeper-3.4.12/data/log

3.1.3.2. 修改zoo.cfg

[root@centoshadoop zookeeper-3.4.12]# cd
/opt/hadoop/zookeeper-3.4.12/conf/

[root@centoshadoop conf]# cp zoo_sample.cfg zoo.cfg

[root@centoshadoop conf]# vi zoo.cfg

tickTime=2000

initLimit=10

syncLimit=5

clientPort=2181

dataDir=/opt/hadoop/zookeeper-3.4.12/data/db

dataLogDir=/opt/hadoop/zookeeper-3.4.12/data/log

#自动清除日志文件

autopurge.snapRetainCount=20

autopurge.purgeInterval=48

#
Zookeeper Cluster

#
server.1=192.168.1.81:2888:3888

#
server.2=192.168.1.82:2888:3888

#
server.3=192.168.1.83:2888:3888

3.2.  HBase

3.2.1.  HBase下载

https://hbase.apache.org/downloads.html

3.2.2.  HBase解压

将下载的apache-hive-2.3.3-bin.tar.gz解压到/opt/hadoop/hbase-2.0.0目录下

4.  配置

4.1.  修改profile文件

vi
/etc/profile

# export
JAVA_HOME=/opt/java/jdk1.7.0_45

export
JAVA_HOME=/opt/java/jdk1.8.0_171

# export
CLASSPATH=.:$JAVA_HOME/lib

export
HADOOP_HOME=/opt/hadoop/hadoop-2.7.5

export
HADOOP_INSTALL=$HADOOP_HOME

export
HADOOP_MAPRED_HOME=$HADOOP_HOME

export
HADOOP_COMMON_HOME=$HADOOP_HOME

export
HADOOP_HDFS_HOME=$HADOOP_HOME

export
YARN_HOME=$HADOOP_HOME

export
HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native

export
PIG_HOME=/opt/hadoop/pig-0.17.0

export
HIVE_HOME=/opt/hadoop/hive-2.3.3

export
PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

export
PATH=$PATH:$PIG_HOME/bin

export
PATH=$PATH:$HIVE_HOME/bin

export
SQOOP_HOME=/opt/hadoop/sqoop-1.99.7

export
PATH=$PATH:$SQOOP_HOME/bin

export
CLASSPATH=$CLASSPATH:$SQOOP_HOME/server/lib:$SQOOP_HOME/shell/lib:$SQOOP_HOME/tools/lib

export
SQOOP_SERVER_EXTRA_LIB=/opt/hadoop/sqoop-1.99.7/jdbcDriver/

export
HBASE_HOME=/opt/hadoop/hbase-2.0.0

export
PATH=$PATH:$HBASE_HOME/bin

export
CLASSPATH=.:$JAVA_HOME/lib:$HIVE_HOME/lib

export
LD_LIBRARY_PATH=$JAVA_HOME/jre/lib/amd64/server:/usr/local/lib:$HADOOP_HOME/lib/native

export
JAVA_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_LIBRARY_PATH

export
HADOOP_USER_NAME=hadoop

4.2.  将JDK升级为1.8版本

将JDK切换成1.8的版本，并修改所有与JAVA_HOME相关的变量

4.3.  修改HBase的配置文件

4.3.1.  创建目录

[root@centoshadoop hbase-2.0.0]# mkdir -p
/opt/hadoop/hbase-2.0.0/data/tmp

[root@centoshadoop hbase-2.0.0]# mkdir -p
/opt/hadoop/hbase-2.0.0/data/tmp/zookeeper

4.3.2.  修改hbase-env.sh

[root@centoshadoop conf]# vi
/opt/hadoop/hbase-2.0.0/conf/hbase-env.sh

export JAVA_HOME=/opt/java/jdk1.8.0_171

export HBASE_MANAGES_ZK=false

4.3.3.  修改hbase-site.xml

[root@centoshadoop conf]# vi
/opt/hadoop/hbase-2.0.0/conf/hbase-site.xml

<property>

<name>hbase.rootdir</name>

<value>hdfs://192.168.1.80:9000/hbase</value>

</property>

<property>

<name>hbase.cluster.distributed</name>

<value>true</value>

</property>

<property>

<name>hbase.zookeeper.quorum</name>

<value>192.168.1.80</value>

</property>

<property>

<name>hbase.temp.dir</name>

<value>/opt/hadoop/hbase-2.0.0/data/tmp</value>

</property>

<property>

<name>hbase.zookeeper.property.dataDir</name>

<value>/opt/hadoop/hbase-2.0.0/data/tmp/zookeeper</value>

</property>

<property>

<name>hbase.zookeeper.property.clientPort</name>

<value>2181</value>

</property>

<property>

<name>hbase.master.info.port</name>

<value>60010</value>

</property>

4.3.4.  修改regionservers

[root@centoshadoop conf]# vi
/opt/hadoop/hbase-2.0.0/conf/regionservers

192.168.1.80

5.  启动Hadoop

5.1.  启动YARN与HDFS

cd
/opt/hadoop/hadoop-2.7.5/sbin

start-all.sh

5.2.  启动historyserver

cd
/opt/hadoop/hadoop-2.7.5/sbin

mr-jobhistory-daemon.sh start historyserver

6.  启动Zookeeper

[root@centoshadoop bin]# cd /opt/hadoop/zookeeper-3.4.12/bin/

[root@centoshadoop bin]# ./zkServer.sh start

ZooKeeper JMX enabled by default

Using
config: /opt/hadoop/zookeeper-3.4.12/bin/../conf/zoo.cfg

Starting
zookeeper ... STARTED

[root@centoshadoop bin]#

7.  启动Hbase

[root@centoshadoop bin]# cd /opt/hadoop/hbase-2.0.0/bin/

[root@centoshadoop bin]# ./start-hbase.sh

[root@centoshadoop bin]# jps

41984
DataNode

42192
SecondaryNameNode

45920
QuorumPeerMain

46643
Jps

42484
NodeManager

46294
HRegionServer

41850
NameNode

46170
HMaster

29245
JobHistoryServer

42365
ResourceManager

[root@centoshadoop bin]#

8.  应用HBase工具

8.1.  测试

8.1.1.  用浏览器访问Hbase状态信息

http://192.168.1.80:16030/

8.2.  HBase的交互式Shell

名称	命令表达式
创建表	create '表名称', '列名称1','列名称2','列名称N'
添加记录	put '表名称', '行名称', '列名称:', '值'
查看记录	get '表名称', '行名称'
查看表中的记录总数	count  '表名称'
删除记录	delete  '表名' ,'行名称' , '列名称'
删除一张表	先要屏蔽该表，才能对该表进行删除， 第一步disable '表名称' 第二步  drop '表名称'
查看所有记录	scan "表名称"
查看某个表某个列中所有数据	scan "表名称" , ['列名称:']
更新记录	就是重写一遍进行覆盖

8.2.1.  启动运行HBase的交互式Shell环境

[root@centoshadoop bin]# cd /opt/hadoop/hbase-2.0.0/bin/

[root@centoshadoop bin]# ./hbase shell

8.2.2.  创建表

hbase(main):004:0> create 'test', 'data'

Created
table test

Took
1.4466 seconds

=>
Hbase::Table - test

hbase(main):005:0>

test:
表名

data:
列族（column family）

8.2.3.  显示表

hbase(main):005:0> list

TABLE

test

1
row(s)

Took
0.0250 seconds

=>
["test"]

8.2.4.  加入数据

hbase(main):006:0> put 'test', 'row1', 'data:1',
'value1'

Took
0.1997 seconds

hbase(main):007:0> put 'test', 'row2', 'data:2',
'value2'

Took
0.0140 seconds

hbase(main):008:0> put 'test', 'row3', 'data:3',
'value3'

Took
0.0061 seconds

hbase(main):009:0>

test：表名

row1：行键（row key）

data:1：列（column）

8.2.5.  查询数据

hbase(main):009:0> scan 'test'

ROW                           COLUMN+CELL

row1                         column=data:1, timestamp=1527136702544, value=value1

row2                         column=data:2, timestamp=1527136720577, value=value2

row3                         column=data:3,
timestamp=1527136732695, value=value3

3
row(s)

Took
0.0360 seconds

8.2.6.  删除表