storm环境搭建(前言)—— 翻译 Setting Up a Development Environment
Setting Up a Development Environment
搭建storm开发环境所需步骤:
- Download a Storm release , unpack it, and put the unpacked
bin/directory on your PATH.(下载storm的release版本, 解压,并且把bin/目录加到环境变量PATH里面去。) - To be able to start and stop topologies on a remote cluster, put the cluster information in
为了让我们可以启动/停止远端storm集群上的topology, 把集群的信息配置在~/.storm/storm.yaml. (~/.storm/storm.yaml里面。)
具体参见:storm环境搭建
What is a development environment?(What 即什么样子)
Storm has two modes of operation: local mode and remote mode. In local mode, you can develop and test topologies completely in process on your local machine. In remote mode, you submit topologies for execution on a cluster of machines.
A Storm development environment has everything installed so that you can develop and test Storm topologies in local mode, package topologies for execution on a remote cluster, and submit/kill topologies on a remote cluster.
Let's quickly go over the relationship between your machine and a remote cluster.(你的机器与远端集群的关系)
A Storm cluster is managed by a master node called "Nimbus". Your machine communicates with Nimbus to submit code (packaged as a jar) and topologies for execution on the cluster, and Nimbus will take care of distributing that code around the cluster and assigning workers to run your topology. Your machine uses a command line client called storm to communicate with Nimbus. The storm client is only used for remote mode; it is not used for developing and testing topologies in local mode.
storm集群是被一个称作Nimbus的控制节点所管理的。你的机器与nimbus通信以提交topology的代码,运行这个topology,而nimbus会自动在集群内部分发你的topology代码,分配任务给各个机器。你的机器使用一个称为storm的客户端去和nimbus通信。storm只有在远程模式的时候才有用; 对于用本地模式开发、测试topology来说是没什么用的。
Installing a Storm release locally
If you want to be able to submit topologies to a remote cluster from your machine, you should install a Storm release locally. Installing a Storm release will give you the storm client that you can use to interact with remote clusters. To install Storm locally, download a release from here and unzip it somewhere on your computer. Then add the unpacked bin/directory onto your PATH and make sure the bin/storm script is executable.
Installing a Storm release locally is only for interacting with remote clusters. For developing and testing topologies in local mode, it is recommended that you use Maven to include Storm as a dev dependency for your project. You can read more about using Maven for this purpose on Maven.
Starting and stopping topologies on a remote cluster
The previous step installed the storm client on your machine which is used to communicate with remote Storm clusters. Now all you have to do is tell the client which Storm cluster to talk to. To do this, all you have to do is put the host address of the master in the ~/.storm/storm.yaml file. It should look something like this:
nimbus.host: "123.45.678.890"
Alternatively, if you use the storm-deploy project to provision Storm clusters on AWS, it will automatically set up your ~/.storm/storm.yaml file. You can manually attach to a Storm cluster (or switch between multiple clusters) using the "attach" command, like so:
lein run :deploy --attach --name mystormcluster
More information is on the storm-deploy wiki
storm环境搭建(前言)—— 翻译 Setting Up a Development Environment的更多相关文章
- storm环境搭建
备注——使用: 1.单机版本: 启动zkServer.nimbus.supervisor.ui服务: zkServer.sh start zkServer.sh status #查看zkserver是 ...
- 【原】Storm环境搭建
2.Storm环境搭建 单机 ... 集群 ... 搭建Storm开发环境 搭建Storm开发环境主要概括为以下两步: 1.下载Storm发行稳定版,然后解压,最后把解压后的bin/文件所在目录添加到 ...
- Storm环境搭建(分布式集群)
作为流计算的开篇,笔者首先给出storm的安装和部署,storm的第二篇,笔者将详细的介绍storm的工作原理.下边直接上干货,跟笔者的步伐一块儿安装storm. 原文链接:Storm环境搭建(分布式 ...
- 170824、storm 环境搭建
/*********************storm 环境搭建 **********************/ 1 安装jdk1.7 2 安装zookeeper集群环境 3 安装Python2.6. ...
- 《OD大数据实战》Storm环境搭建
一.环境搭建 1. 下载 http://www.apache.org/dyn/closer.lua/storm/apache-storm-0.9.6/apache-storm-0.9.6.tar.gz ...
- 54.Storm环境搭建
集群环境搭建 关闭防火墙,修改/etc/hosts配置(3台机器的ip可以相互通信) 下载安装jdk7(1.6以上),配置JAVA_HOME, CLASSPATH 搭建Zookeeper集群(保证3台 ...
- Jquery环境搭建前言
- apache - storm - Setting Up a Development Environment
Installing a Storm release locally If you want to be able to submit topologies to a remote cluster f ...
- [Django] Setting up Django Development Environment in Ubuntu 14.04
1. Python Of course you will need Python. Still Python 2.7 is preferred, however if you would like t ...
随机推荐
- How to install OpenResty
How to install OpenResty 15 January 2014, 6:18 am OpenResty, also called “ngx_openresty”, is a web ...
- JetBrains WebStorm 7.0 Build 131.202 Win/Mac/Liniux
JetBrains WebStorm 7.0 Build 131.202 (Win/Mac/Liniux) | 121.6/106/133 Mb WebStorm 7 — Everything you ...
- JS的trim()方法
去除字符串左右两端的空格,在vbscript里面可以轻松地使用 trim.ltrim 或 rtrim,但在js中却没有这3个内置方法,需要手工编写.下面的实现方法是用到了正则表达式,效率不错,并把这三 ...
- 一个很不错的适合PHPER们书单,推荐给大家【转】
来我博客的访客们中,有一些是PHP的初学者,是不是很迷茫PHP应该怎么学?应该买什么样的书?到处问人,到处求助?这下好了. 正好看到黑夜路人在博客上推荐了一个书单,看上去都非常不错,很多我也没有读过, ...
- SQL SERVER 中的事务
所谓事务是用户定义的一个数据库操作序列,这些操作要么全做要么全不做,是一个不可分割的工作单位.例如,在关系数据库中,一个事务可以是一条SQL语句.一组SQL语句或整个程序. 简单举个例子就是 你要同时 ...
- C++复数四则运算的实现
程序主要实现复数的加减乘,数乘,取共轭功能. 将所有函数都定义为了成员函数. 使用库函数atof将字符串转换为浮点型数据. 函数主要难点在于处理输入.由于需要判断输入是选择退出还是继续,所以用字符串来 ...
- 4.在二元树中找出和为某一值的所有路径[FindPathsInBinaryTree]
[题目]: 输入一个整数和一棵二元树.从树的根结点开始往下访问一直到叶结点所经过的所有结点形成一条路径.打印出和与输入整数相等的所有路径. 例如输入整数22和如下二元树 10 ...
- ShortestPath:Six Degrees of Cowvin Bacon(POJ 2139)
牛与电影 题目大意:就是一群牛,又在玩游戏了(怎么你们经常玩游戏),这个游戏规则如下,把牛拆分成一个一个组,并且定义一个“度”,规定在一个组的牛与他自己的度为0,与其他牛的度为1,不同组的牛不存在度, ...
- BestCoder17 1001.Chessboard(hdu 5100) 解题报告
题目链接:http://acm.hdu.edu.cn/showproblem.php?pid=5100 题目意思:有一个 n * n 的棋盘,需要用 k * 1 的瓷砖去覆盖,问最大覆盖面积是多少. ...
- BestCoder12 1002.Help him(hdu 5059) 解题报告
题目链接:http://acm.hdu.edu.cn/showproblem.php?pid=5059 题目意思:就是输入一行不多于 100 的字符串(除了'\n' 和 '\r' 的任意字符),问是否 ...