Mac OS Storm＋Kafka+Zookeeper配置

先补充一个前两天mac配置的文档。

首先确定由jdk scala环境

JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home
export PATH=${PATH}:${JAVA_HOME}/bin
SCALA_HOME=/Users/lu/develop/dev/scala-2.10.3
export PATH=${PATH}:${SCALA_HOME}/bin

zookeeper

wget http://mirrors.cnnic.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
tar zxvf zookeeper-3.4.6.tar.gz
cd zookeeper-3.4.6
cp -rf conf/zoo_sample.cfg conf/zoo.cfg

修改zoo.cfg的dataDir

dataDir=/Users/lu/develop/bigdata/zookeeper-3.4.6/zkdata

kafka配置，单机其实没什么配置的，直接按官方链接,下载解压

tar zxvf kafka_2.10-0.8.2-beta.tgz

启动

cd kafka_2.10-0.8.2-beta
bin/kafka-server-start.sh config/server.properties

storm配置, 这里本来打算用还未release的0.9.3,发现zip,gz都不可用，最后只好用了0.9.3-rc1版, 前天其实又release了

storm.yaml配置

storm.zookeeper.servers:
 - "127.0.0.1"
nimbus.host: "127.0.0.1"
storm.messaging.transport: "backtype.storm.messaging.netty.Context"
storm.messaging.netty.server_worker_threads: 1
storm.messaging.netty.client_worker_threads: 1
storm.messaging.netty.buffer_size: 5242880
storm.messaging.netty.max_retries: 100
storm.messaging.netty.max_wait_ms: 1000
storm.messaging.netty.min_wait_ms: 100

配置Storm home

export STORM_HOME=/Users/lu/develop/bigdata/storm-0.9.3
export PATH=$PATH:$STORM_HOME/bin

可以启动起来了,这里kafka 不用自带的启动zk

后续：

1，开始flume+kafka+storm+mysql/hbase 其实storm+hbase 现在很方便了，storm 自带了。kafka+storm也是

2，一些笔记triden, ml,state,filter等