Hbase-2.0.0_01_安装部署

该文章是基于 Hadoop2.7.6_01_部署 进行的

1. 主机规划

主机名称	IP信息	内网IP	操作系统	安装软件	备注：运行程序
mini01	10.0.0.11	172.16.1.11	CentOS 7.4	jdk、hadoop、zookeeper、hbase	QuorumPeerMain、NameNode、HMaster
mini02	10.0.0.12	172.16.1.12	CentOS 7.4	jdk、hadoop、zookeeper、hbase	QuorumPeerMain、ResourceManager、HMaster
mini03	10.0.0.13	172.16.1.13	CentOS 7.4	jdk、hadoop、zookeeper、hbase	QuorumPeerMain、DataNode、NodeManager、HRegionServer
mini04	10.0.0.14	172.16.1.14	CentOS 7.4	jdk、hadoop、zookeeper、hbase	QuorumPeerMain、DataNode、NodeManager、HRegionServer
mini05	10.0.0.15	172.16.1.15	CentOS 7.4	jdk、hadoop、zookeeper、hbase	QuorumPeerMain、DataNode、NodeManager、HRegionServer

2. Zookeeper部署

共部署5台，所以在mini01~mini05都得部署

2.1. 配置信息

 [yun@mini01 conf]$ pwd
 /app/zookeeper/conf
 [yun@mini01 conf]$ cat zoo.cfg
 #单个客户端与单台服务器之间的连接数的限制，是ip级别的，默认是60，如果设置为0，那么表明不作任何限制。
 maxClientCnxns=
 # The number of milliseconds of each tick
 tickTime=
 # The number of ticks that the initial
 # synchronization phase can take
 initLimit=
 # The number of ticks that can pass between
 # sending a request and getting an acknowledgement
 syncLimit=
 # the directory where the snapshot is stored.
 # do not use /tmp for storage, /tmp here is just
 # example sakes.
 # dataDir=/tmp/zookeeper
 dataDir=/app/bigdata/zookeeper/data
 # the port at which the clients will connect
 clientPort=
 #
 # Be sure to read the maintenance section of the
 # administrator guide before turning on autopurge.
 #
 # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
 #
 # The number of snapshots to retain in dataDir
 #autopurge.snapRetainCount=
 # Purge task interval in hours
 # Set to "" to disable auto purge feature
 #autopurge.purgeInterval=

 # leader和follow通信端口和投票选举端口
 server.=mini01::
 server.=mini02::
 server.=mini03::
 server.=mini04::
 server.=mini05::

2.2. 添加myid文件

 [yun@mini01 data]$ pwd
 /app/bigdata/zookeeper/data
 [yun@mini01 data]$ ll
 total
 -rw-r--r--  yun yun    May  : myid
 drwxr-xr-x  yun yun  Jun  : version-
 [yun@mini01 data]$ cat myid  # 其中mini01的myid 为1；mini02的myid 为2；mini03的myid 为3；mini04的myid 为4；mini05的myid 为5
 

2.3. 环境变量

 [root@mini01 profile.d]# pwd
 /etc/profile.d
 [root@mini01 profile.d]# cat zk.sh
 export ZK_HOME="/app/zookeeper"
 export PATH=$ZK_HOME/bin:$PATH

 [root@mini01 profile.d]# logout
 [yun@mini01 conf]$ source /etc/profile # 重新加载环境变量  

2.4. 启动zk服务

 # 依次在启动mini01、mini02、mini03、mini04、mini05  zk服务
 [yun@mini01 zookeeper]$ pwd
 /app/zookeeper
 [yun@mini01 zookeeper]$ zkServer.sh start
 JMX enabled by default
 Using config: /app/zookeeper/bin/../conf/zoo.cfg
 Starting zookeeper ... STARTED

建议在 /app/zookeeper，因为在启动时，会有相关日志产生

 [yun@mini01 zookeeper]$ pwd
 /app/zookeeper
 [yun@mini01 zookeeper]$ ll zookeeper.out
 -rw-rw-r--  yun yun  Aug   : zookeeper.out

2.5. 查询运行状态

 # 其中mini01、mini02、mini04、mini05状态如下
 [yun@mini01 zookeeper]$ zkServer.sh status
 JMX enabled by default
 Using config: /app/zookeeper/bin/../conf/zoo.cfg
 Mode: follower

 # 其中mini03 状态如下
 [yun@mini03 zookeeper]$ zkServer.sh status
 JMX enabled by default
 Using config: /app/zookeeper/bin/../conf/zoo.cfg
 Mode: leader

3. Hbase部署与配置修改

3.1. 软件部署

 [yun@mini01 software]$ pwd
 /app/software
 [yun@mini01 software]$ tar xf hbase-2.0.-bin.tar.gz
 [yun@mini01 software]$ mv hbase-2.0. /app/
 [yun@mini01 software]$ cd
 [yun@mini01 ~]$ ln -s hbase-2.0./ hbase

3.2. 环境变量

注意所有部署hbase的机器【mini01、mini02、mini03、mini04、mini05】都需要该环境变量

 [root@mini01 profile.d]# pwd
 /etc/profile.d
 [root@mini01 profile.d]# cat hbase.sh  # 也可以直接写在 /etc/profile 文件中
 export HBASE_HOME="/app/hbase"
 export PATH=$HBASE_HOME/bin:$PATH

 [root@mini01 profile.d]# logout
 [yun@mini01 hbase]$ source /etc/profile  # 使用yun用户，并重新加载环境变量 

3.3. hbase-env.sh 修改

 [yun@mini01 conf]$ pwd
 /app/hbase/conf
 [yun@mini01 conf]$ cat hbase-env.sh
 #!/usr/bin/env bash
 ………………
 # The java implementation to use.  Java 1.8+ required.
 # export JAVA_HOME=/usr/java/jdk1.8.0/
 export JAVA_HOME=${JAVA_HOME}

 # Extra Java CLASSPATH elements.  Optional.  # hadoop配置文件的位置
 # export HBASE_CLASSPATH=
 export HBASE_CLASSPATH=${HADOOP_HOME}/etc/hadoop/
 ………………
 # Tell HBase whether it should manage it's own instance of ZooKeeper or not.
 # 如果使用独立安装的zookeeper这个地方就是false
 # export HBASE_MANAGES_ZK=true
 export HBASE_MANAGES_ZK=false
 ………………

3.4. hbase-site.xml 修改

 [yun@mini01 conf]$ pwd
 /app/hbase/conf
 [yun@mini01 conf]$ cat hbase-site.xml
 <?xml version="1.0"?>
 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
 <!--
 /**
  *
 ………………
  */
 -->
 <configuration>
   <property>
     <name> hbase.master.port</name>  <!-- hbasemaster的端口 -->
     <value></value>
   </property>
   <property>
     <name>hbase.tmp.dir</name>        <!-- hbase 临时存储 -->
     <value>/app/hbase/tmp</value>
   </property>
   <property>
     <name>hbase.master.maxclockskew</name>   <!-- 时间同步允许的时间差 单位毫秒 -->
     <value></value>
   </property>
   <property>
     <name>hbase.rootdir</name>
     <value>hdfs://mini01:9000/hbase</value>  <!-- hbase共享目录，持久化hbase数据  存放在对应的HDFS上 -->
   </property>
   <property>
     <name>hbase.cluster.distributed</name>  <!-- 是否分布式运行，false即为单机 -->
     <value>true</value>
   </property>
   <property>
     <name>hbase.zookeeper.property.clientPort</name> <!-- zookeeper端口 -->
     <value></value>
   </property>
   <property>
     <name>hbase.zookeeper.quorum</name>  <!-- zookeeper地址 -->
     <value>mini01,mini02,mini03,mini04,mini05</value>
   </property>
   <property>
     <name>hbase.zookeeper.property.dataDir</name>  <!-- zookeeper配置信息快照的位置 -->
     <value>/app/hbase/tmp/zookeeper</value>
   </property>
 </configuration>

3.5. regionservers 修改

 [yun@mini01 conf]$ pwd
 /app/hbase/conf
 [yun@mini01 conf]$ cat regionservers  # 从机器的域名
 mini03
 mini04
 mini05

4. Hbase的分发与启动

注意：启动hbase之前，必须保证hadoop集群和zookeeper集群是可用的。

4.1. hbase分发到其他机器

将 /app/hbase-2.0.0 从mini01 分发到mini02【用于HA】、mini03、mini04、mini05

其中配置不需要任何修改

 scp hbase-2.0. yun@mini02:/app
 scp hbase-2.0. yun@mini03:/app
 scp hbase-2.0. yun@mini04:/app
 scp hbase-2.0. yun@mini05:/app

分发完毕后，记得登录不同的主机然后创建软连接

 [yun@mini02 ~]$ pwd
 /app
 [yun@mini02 ~]$ ln -s hbase-2.0./ hbase

4.2. 启动程序

 [yun@mini01 ~]$ start-hbase.sh
 SLF4J: Class path contains multiple SLF4J bindings.
 SLF4J: Found binding in [jar:file:/app/hbase-2.0./lib/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
 SLF4J: Found binding in [jar:file:/app/hadoop-2.7./share/hadoop/common/lib/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 running master, logging to /app/hbase/logs/hbase-yun-master-mini01.out
 SLF4J: Class path contains multiple SLF4J bindings.
 SLF4J: Found binding in [jar:file:/app/hbase-2.0./lib/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
 SLF4J: Found binding in [jar:file:/app/hadoop-2.7./share/hadoop/common/lib/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 mini05: running regionserver, logging to /app/hbase/bin/../logs/hbase-yun-regionserver-mini05.out
 mini04: running regionserver, logging to /app/hbase/bin/../logs/hbase-yun-regionserver-mini04.out
 mini03: running regionserver, logging to /app/hbase/bin/../logs/hbase-yun-regionserver-mini03.out

master进程

 [yun@mini01 ~]$ jps
  SecondaryNameNode
  NameNode
  HMaster
  QuorumPeerMain
  Jps

slave进程

 [yun@mini04 ~]$ jps
  Jps
  QuorumPeerMain
  NodeManager
  DataNode
  HRegionServer

4.3. zk中的信息

 [zk: localhost:(CONNECTED) ] ls /hbase
 [replication, meta-region-server, rs, splitWAL, backup-masters, table-lock, flush-table-proc, master-maintenance, online-snapshot, switch, master, running, draining, namespace, hbaseid, table]

 [zk: localhost:(CONNECTED) ] ls /hbase/rs
 [mini03,,, mini05,,, mini04,,]

4.4. 浏览器访问

 http://mini01:16010    

5. Hbase的HA

根据规划，mini01和mini02为Hmaster，其中mini01的Hmaster已经起来了

5.1. 启动另一个HMaster

 [yun@mini02 ~]$ hbase-daemon.sh start master  # 在mini02起一个 HMaster
 running master, logging to /app/hbase/logs/hbase-yun-master-mini02.out
 SLF4J: Class path contains multiple SLF4J bindings.
 SLF4J: Found binding in [jar:file:/app/hbase-2.0./lib/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
 SLF4J: Found binding in [jar:file:/app/hadoop-2.7./share/hadoop/common/lib/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
 SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
 SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
 [yun@mini02 ~]$
 [yun@mini02 ~]$ jps
  Jps
  ResourceManager
  QuorumPeerMain
  HMaster

5.2. zk中的信息

 [zk: localhost:(CONNECTED) ] ls /hbase/backup-masters
 [mini02,,]

5.3. 浏览器访问

 http://mini02:16010