HADOOP (十一).安装hbase

下载安装包并解压
设置hbase环境变量
配置hbase-site.xml
启动hbase
检测hbase启动情况
测试hbase shell

下载安装包并解压

https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/1.3.1/hbase-1.3.1-bin.tar.gz
…..
[hbase@hadoop1 opt]$ tar -zxvf hbase-1.3.1-bin.tar.gz

设置hbase环境变量

[hbase@hadoop1 opt]$ cd hbase-1.3.1/conf/
[hbase@hadoop1 conf]$ vi hbase-env.sh
#### 看情况设置以下环境变量:
#export JAVA_HOME=/usr/java/jdk1.6.0/
#export HBASE_HEAPSIZE=1G #堆内存
#export HBASE_OPTS="-XX:+UseConcMarkSweepGC"  #jvm启动参数
#export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m" #hmaster
#export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m"  #hregionserver
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps"  #gc相关
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M"  #java 参数不太懂,自行修正
…………….

参数较多,请逐个检查.

最重要的就是JAVA_HOME,但我在/etc/profile里设置了,这里就不管了.

由于是做实验,这里我就设置一下log目录.关于hbase-evn.sh的设置,以后会详细讲解.

export HBASE_LOG_DIR=/var/log/hbase/

注意:如果没有在/etc/profile中设置HADOOP_CONF_DIR,则需要在hbase-evn.sh中设置HADOOP_CONF_DIR,否则hbase读不到hdfs-site.xml,无法找到hdfs.或者将hdfs-site.xml复制到$HBASE_HOME/conf/下也行.这里的hdfs-site.xml用客户端的配置即可.

配置hbase-site.xml

<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://mycluster/hbase</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>hadoop3,hadoop4,hadoop5</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/hbase</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <property>
    <name>HBASE_MANAGES_ZK</name>
    <value>false</value>
  </property>
</configuration>

上面的参数意思分别是:

hbase在hdfs的主目录

zookeeper服务器节点

hbase在zookeeper中的目录

hbase是否是分布式的

是否用hbase自带的zookeeper

启动hbase

首先使用hdfs账号为hbase授权:

[hdfs@hadoop2 root]$ hdfs dfs -mkdir /hbase
[hdfs@hadoop2 root]$ hdfs dfs -chown hbase:hbase /hbase

在hadoop1上启动master:

[hbase@hadoop1 hbase-1.3.1]$ bin/hbase-daemon.sh start master

在hadoop3 hadoop4 hadoop5上启动hreigonserver:

[hbase@hadoop3 hbase-1.3.1]$ bin/hbase-daemon.sh start regionserver

在需要的节点上启动master备份节点:

bin/hbase-daemon.sh start master --backup

检测hbase启动情况

1.jps检查

[hbase@hadoop1 hbase-1.3.1]$ jps
25914 HMaster

[hbase@hadoop3 hbase-1.3.1]$ jps
12623 HRegionServe

看到master和regionserver都启动了

2.打开hbase web

打开:http://hadoop1:16010/



发现没有一个hregoinser连接上master!!!

好吧,打开/var/log/hbase/下的regionserver日志发现:

2017-06-27 23:16:51,039 FATAL [regionserver/hadoop3/192.168.0.12:16020] regionserver.HRegionServer: Master rejected startup because clock is out of sync
org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server hadoop3,16020,1498576600617 has been rejected; Reported time is too far out of sync with master.  Time difference of 54158ms > max allowed of 30000ms
    at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:410)
    at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:276)
    at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:363)
    at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:8615)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
    at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
    at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:332)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2337)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:929)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ClockOutOfSyncException): org.apache.hadoop.hbase.ClockOutOfSyncException: Server hadoop3,16020,1498576600617 has been rejected; Reported time is too far out of sync with master.  T
ime difference of 54158ms > max allowed of 30000ms
    at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:410)
    at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:276)

原因是regionser和master的时间差太多了!又检查了一下,发现机器上的ntp服务挂了!!

设置各机器的时间:

date -s '2017-07-06 21:36:40'

再次启动各进程,打开web页面,发现连接上master了:

测试hbase shell

[hbase@hadoop5 hbase-1.3.1]$ bin/hbase shell
hbase(main):001:0> create 'test','cf1'
0 row(s) in 4.9050 seconds

=> Hbase::Table - test
hbase(main):002:0> list
TABLE
test
hbase(main):004:0> put 'test', 'row1', 'cf1:a', 'value1'
0 row(s) in 0.0170 seconds

hbase(main):005:0> put 'test', 'row2', 'cf1:b', 'value2'
0 row(s) in 0.0090 seconds
hbase(main):012:0* scan test
ArgumentError: wrong number of arguments (0 for 2)

hbase(main):013:0> scan 'test'
ROW                                                                  COLUMN+CELL
 row1                                                                column=cf1:a, timestamp=1499349174471, value=value1
 row2                                                                column=cf1:b, timestamp=1499349174533, value=value2
2 row(s) in 0.1140 seconds

说明hbase能正常使用

来自为知笔记(Wiz)