使用Erlang和Thrift，与Hbase通信（转）

操作系统是Ubuntu Server 12.10

先安装Thrift

sudo apt-get install libboost-dev libboost-test-dev \

 libboost-program-options-dev libevent-dev automake \

 libtool flex bison pkg-config g++ libssl-dev

如果你还要使用别的语言，也需要安装对应的包

Ruby
- ruby-full ruby-dev librspec-ruby rake rubygems libdaemons-ruby libgemplugin-ruby mongrel
Python
- python-dev python-twisted
Perl
- libbit-vector-perl
Php, install
- php5-dev php5-cli
C_glib
- libglib2.0-dev
Erlang
- erlang-base erlang-eunit erlang-dev
Csharp
- mono-gmcs libmono-dev libmono-system-web2.0-cil
Haskell
- ghc6 cabal-install libghc6-binary-dev libghc6-network-dev libghc6-http-dev

cd thrift

./bootstrap.sh

./configure --with-boost=/usr/local

make

make install

安装Hbase

修改源码包中的conf/hbase-site.xml文件。替换成以下内容：


    hbase.rootdir

    file:///DIRECTORY/hbase

    hbase.zookeeper.property.dataDir

    /DIRECTORY/zookeeper

替换上文中对应的目录位置修改conf/hbase-env.sh 添加JAVA_HOME export JAVA_HOME=/usr/local/jdk 启动Hbase

$ ./bin/start-hbase.sh

starting Master, logging to logs/hbase-user-master-example.org.out

$ ./bin/hbase shell

HBase Shell; enter 'help' for list of supported commands.

Type "exit" to leave the HBase Shell

Version 0.94.2, r1395367, Sun Oct  7 19:11:01 UTC 2012

hbase(main):001:0>

hbase(main):001:0> create 't1','cf'

0 row(s) in 1.6600 seconds

hbase(main):002:0> list

TABLE

t1

1 row(s) in 0.0130 seconds

hbase(main):004:0> put 't1', 'row1', 'cf:a', 'value1'

0 row(s) in 0.0330 seconds

hbase(main):005:0> put 't1', 'row2', 'cf:a', 'value2'

0 row(s) in 0.0040 seconds

hbase(main):006:0> put 't1', 'row3', 'cf:a', 'value3'

0 row(s) in 0.0040 seconds

hbase(main):007:0> scan 't1'

ROW                        COLUMN+CELL

 row1                      column=cf:a, timestamp=1352453214504, value=value1

 row2                      column=cf:a, timestamp=1352453236805, value=value2

 row3                      column=cf:a, timestamp=1352453244172, value=value3

3 row(s) in 0.0320 seconds

hbase(main):008:0> get 't1', 'row1'

COLUMN                     CELL

 cf:a                      timestamp=1352453214504, value=value1

1 row(s) in 0.0120 seconds

hbase(main):009:0> disable 't1'

0 row(s) in 2.0510 seconds

hbase(main):011:0> drop 't1'

0 row(s) in 1.0480 seconds

hbase(main):012:0> exit

如果上述测试脚本都能正常执行，就证明你Hbase安装成功了。注意，如果你使用ubuntu，当执行命令的时候，提示

ERROR: org.apache.hadoop.hbase.PleaseHoldException: org.apache.hadoop.hbase.PleaseHoldException: Master is initializing

在logs/hbase-*-ubuntu.log中显示类似

Failed setting up proxy interface org.apache.hadoop.hbase.ipc.HRegionInterface to localhost/127.0.0.1:55915 after attempts=1

Caused by: java.net.ConnectException: Connection refused

出现这种情形，请检查/etc/hosts文件，把 127.0.1.1 ubuntu 改成 127.0.0.1 ubuntu 之后重新启动Hbase就可以正常运行了。

通过Thrift操作Hbase

生产Thrift客户端

mkdir ~/thrift

cd ~/thrift

thrift –gen erl [hbase-root]/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift

启动两个客户端

$ cd ~/hbase

$ ./bin/start-hbase.sh

在另外一个客户端,启动Thrift Server

$ cd ~/hbase

elton@ubuntu:~/hbase$ ./bin/hbase thrift start

¹²⁄₁₁/09 17:41:33 INFO util.VersionInfo: HBase 0.94.2

…

¹²⁄₁₁/09 17:41:35 DEBUG thrift.ThriftServerRunner: Using binary protocol

¹²⁄₁₁/09 17:41:35 INFO thrift.ThriftServerRunner: starting TBoundedThreadPoolServer on /0.0.0.0:9090; min worker threads=16, max worker threads=1000, max queued requests=1000

之后就可以产生一个gen-erl，进入它，编译其中的Erlang源文件

cd gen-erl

erlc *.erl

在gen-erl目录下，进入erlang shell。

$ erl

Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:4:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.9.2 (abort with ^G) 1> {ok, Client} = thrift_client_util:new(“127.0.0.1”, 9090, hbase_thrift, []). {ok,{tclient,hbase_thrift, {protocol,thrift_binary_protocol, {binary_protocol, {transport,thrift_buffered_transport, {buffered_transport, {transport,thrift_socket_transport, {data,#Port<0.631>,infinity}}, []}}, true,true}}, 0}}

2> thrift_client:call(Client, getTableNames, []). tclient,hbase_thrift, {protocol,thrift_binary_protocol, {binary_protocol, {transport,thrift_buffered_transport, {buffered_transport, {transport,thrift_socket_transport, {data,#Port<0.631>,infinity}}, []}}, true,true}}, 0}, {ok,[]}}

具体的clinet API可以参见 https://github.com/interline/erlang-thrift

转自： http://blog.prosight.me/blogs/1025/