storm集群配置

环境：centos6.4
软件：
jzmq-master-----java与c++通讯的桥梁，有了它，就可以使用zeromp了
storm-0.8.2
zeromq-2.1.7-----号称史上最牛逼的消息队列（用c++写的）
zookeeper-3.4.5

主机配置：

1.修改主机名：

vim /etc/sysconfig/network

NETWORKING=yes
HOSTNAME=MASTER

2.修改IP：

vim /etc/sysconfig/network-scripts/ifcfg-eth0

BOOTPROTO="static"
HWADDR="00:0C:29:FC:62:B6"
IPV6INIT="yes"
NM_CONTROLLED="yes"
ONBOOT="yes"
TYPE="Ethernet"
UUID="d0731a46-36df-4ab1-9bac-42bb151acc41"
IPADDR="192.168.1.250"
NETMASK="255.255.255.0"
GATEWAY="192.168.1.1"
DNS1="8.8.8.8"
DNS2="8.8.4.4"

3.修改主机和IP的映射关系

vim /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.250   MASTER
192.168.1.251   SLAVE-1
192.168.1.252   SLAVE-2

4.关闭防火墙

vim /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

继续执行命令：

service iptables stop
chkconfig iptables off
reboot

5.上传jdk安装包

6.解压安装

tar -xzvf jdk-8u151-linux-i586.tar.gz

7.配置环境变量

vim /etc/profile

在文件最后面添加：

export JAVA_HOME=/usr/java/jdk1.6.0_45
export CLASSPATH=$JAVA_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin

更新配置：

source /etc/profile

上传zeromq-2.1.7.tar.gz、jzmq-master.zip、storm-0.8.2.zip、zookeeper-3.4.5.tar.gz等文件
1.编译安装ZMQ：

tar -xzf zeromq-2.1.7.tar.gz
cd zeromq-2.1.7
./configure

#编译可能会出错：configure: error: Unable to find a working C++ compiler
#安装一下依赖的rpm包：libstdc++-devel gcc-c++
可以上网的情况下：

yum install gcc-c++

虚拟机不能上网情况：首先到http://mirrors.163.com/centos/6.4/os/x86_64/Packages/ 下载rpm

rpm -i libstdc++-devel-4.4.7-3.el6.x86_64.rpm
rpm -i gcc-c++-4.4.7-3.el6.x86_64.rpm
rpm -i libuuid-devel-2.17.2-12.9.el6.x86_64.rpm

yum install libuuid-devel

./configure
make
make install

2.编译安装JZMQ：

unzip jzmq-master.zip
cd jzmq
./autogen.sh

#报错:autogen.sh: error: could not find libtool. libtool is required to run autogen.sh. 缺少libtool

yum install libtool

或者手动安装

rpm -i autoconf-2.63-5.1.el6.noarch.rpm
rpm -i automake-1.11.1-4.el6.noarch.rpm
rpm -i libtool-2.2.6-15.5.el6.x86_64.rpm

./configure
make
make install

3.编译安装zookeeper：

将zookeeper-3.4.6/conf目录下的zoo_sample.cfg文件拷贝一份，命名为为“zoo.cfg”

tar –zxvf zookeeper-3.4.5.tar.gz
cp zoo_sample.cfg zoo.cfg
./zkServer.sh start

4.下载并解压Storm发布版本
下一步，需要在Nimbus和Supervisor机器上安装Storm发行版本。
1. 下载Storm发行版本
wget https://dl.dropbox.com/u/133901206/storm-0.8.2.zip
2. 解压到安装目录下：

unzip storm-0.8.1.zip

修改配置文件 conf/storm.yaml

storm.zookeeper.servers:
     - "192.168.1.222"
     - "192.168.1.223"
     - "192.168.1.224"
     - "192.168.1.225"
 nimbus.host: "192.168.2.222"
 ui.port:  9090
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
#     - "server1"
#     - "server2"
#
 storm.local.dir: "/home/storm/storm_workdir"
 java.library.path: "/usr/local/lib:/usr/lib"

　　

./storm nimbus
./storm ui > /dev/null 2>&1 &

从机配置

storm.zookeeper.servers:
     - "主机ip地址或者hostname"
 nimbus.host: "主机ip地址或者hostname"
 ui.port:  9090
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
#     - org.mycompany.MyType
#     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
#     - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
#     - "server1"
#     - "server2"
#
 storm.local.dir: "/home/storm/storm_workdir"
 java.library.path: "/usr/local/lib:/usr/lib"

　　

storm supervisor &

　　

执行jps 命令

在没有运行任务时，我们必须应该要看到4个进程：

QuorumPeerMain、nimbus、core、supervisor

使用浏览器:

　　 http://192.168.1.222:9090/ 可以看到Storm UI 界面