Spring整合HBase

Spring

HBase

SHDP

§ 系统环境
§ 配置HBase运行环境
§ 创建Maven项目

§ 系统环境

Ubuntu
- Hadoop 2.7.3
- HBase 1.2.3
- JDK 1.8
Windows
- IDEA 16
- Spring 4.3.2.RELEASE
- Spring Data Hadoop 2.4.0.RELEASE

§ 配置HBase运行环境

Hadoop和HBase都配置运行在Ubuntu虚拟机中，HBase以伪分布式模式运行，使用HDFS作为存储系统。

首先需要安装必须的软件：

$ apt-get install ssh
$ apt-get install rsync

并确保ssh可以不使用口令就能够连接localhost。设置方式如下：

$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ chmod 0600 ~/.ssh/authorized_keys

查看Ubuntu的IP地址，然后编辑/etc/hosts，注释掉其他主机映射内容，将<ip>和<domain>替换为本机IP地址和自定义域：

#127.0.0.1 localhost
#127.0.1.1 <domain>
<ip> <domain> localhost

§ 配置Hadoop

以<HADOOP_DIR>表示Hadoop根目录，编辑<HADOOP_DIR>/etc/hadoop/hadoop-env.sh，修改JAVA_HOME为JDK安装目录。

编辑配置文件<HADOOP_DIR>/etc/hadoop/core-site.xml：

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://<ip>:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/<user>/data</value>
</property>
</configuration>

编辑配置文件<HADOOP_DIR>/etc/hadoop/hdfs-site.xml：

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

§ 配置HBase

以<HBASE_DIR>表示HBase根目录，修改<HBASE_DIR>/conf/hbase-env.sh，添加JAVA_HOME。

编辑配置文件<HBASE_DIR>/conf/hbase-site.xml，替换ip和domain为对应内容：

<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://<ip>:9000/hbase</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value><domain></value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
</configuration>

§ 启动Hadoop和HBase

启动Hadoop前先格式化文件系统：

$ ./<HADOOP_DIR>/bin/hdfs namenode -format

然后启动Hadoop和HBase：

$ ./<HADOOP_DIR>/sbin/start-dfs.sh
$ ./<HBASE_DIR>/bin/start-hbase.sh

可以使用jps命令查看所有启动的进程：

$ jps
2786 NameNode
2914 DataNode
6259 HQuorumPeer
6324 HMaster
3083 SecondaryNameNode
6411 HRegionServer

§ 创建Maven项目

新建Maven项目，所需依赖如下：

<properties>
<slf4j.version>1.7.21</slf4j.version>
<spring.version>4.3.2.RELEASE</spring.version>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-tx</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-hadoop</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework</groupId>
<artifactId>spring-context-support</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-auth</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.2.3</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</dependency>
</dependencies>

将HBase的配置文件hbase-site.xml复制到resources下，新建Spring配置文件applicationContext.xml：

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:hdp="http://www.springframework.org/schema/hadoop"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
http://www.springframework.org/schema/hadoop http://www.springframework.org/schema/hadoop/spring-hadoop.xsd">
<context:annotation-config/>
<context:component-scan base-package="com.sample.hbase"/>
<hdp:configuration resources="hbase-site.xml"/>
<hdp:hbase-configuration configuration-ref="hadoopConfiguration"/>
<bean id="hbaseTemplate" class="org.springframework.data.hadoop.hbase.HbaseTemplate">
<property name="configuration" ref="hbaseConfiguration"/>
</bean>
</beans>

新建测试用例：

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = {"classpath*:applicationContext.xml"})
public class BaseTest {
@Autowired
private HbaseTemplate template;
@Test
public void testFind() {
List<String> rows = template.find("user", "cf", "name", new RowMapper<String>() {
public String mapRow(Result result, int i) throws Exception {
return result.toString();
}
});
Assert.assertNotNull(rows);
}
@Test
public void testPut() {
template.put("user", "1", "cf", "name", Bytes.toBytes("Alice"));
}
}

整合完成。