Setting Up a Development Environment

搭建storm开发环境所需步骤:

  1. Download a Storm release , unpack it, and put the unpacked bin/ directory on your PATH.(下载storm的release版本, 解压,并且把bin/目录加到环境变量PATH里面去。)
  2. To be able to start and stop topologies on a remote cluster, put the cluster information in ~/.storm/storm.yaml. (为了让我们可以启动/停止远端storm集群上的topology, 把集群的信息配置在~/.storm/storm.yaml里面。)

具体参见:storm环境搭建

What is a development environment?(What 即什么样子)

Storm has two modes of operation: local mode and remote mode. In local mode, you can develop and test topologies completely in process on your local machine. In remote mode, you submit topologies for execution on a cluster of machines.

A Storm development environment has everything installed so that you can develop and test Storm topologies in local mode, package topologies for execution on a remote cluster, and submit/kill topologies on a remote cluster.

Let's quickly go over the relationship between your machine and a remote cluster.(你的机器与远端集群的关系)

  A Storm cluster is managed by a master node called "Nimbus". Your machine communicates with Nimbus to submit code (packaged as a jar) and topologies for execution on the cluster, and Nimbus will take care of distributing that code around the cluster and assigning workers to run your topology. Your machine uses a command line client called storm to communicate with Nimbus. The storm client is only used for remote mode; it is not used for developing and testing topologies in local mode.

  storm集群是被一个称作Nimbus的控制节点所管理的。你的机器与nimbus通信以提交topology的代码,运行这个topology,而nimbus会自动在集群内部分发你的topology代码,分配任务给各个机器。你的机器使用一个称为storm的客户端去和nimbus通信。storm只有在远程模式的时候才有用; 对于用本地模式开发、测试topology来说是没什么用的。

Installing a Storm release locally

If you want to be able to submit topologies to a remote cluster from your machine, you should install a Storm release locally. Installing a Storm release will give you the storm client that you can use to interact with remote clusters. To install Storm locally, download a release from here and unzip it somewhere on your computer. Then add the unpacked bin/directory onto your PATH and make sure the bin/storm script is executable.

Installing a Storm release locally is only for interacting with remote clusters. For developing and testing topologies in local mode, it is recommended that you use Maven to include Storm as a dev dependency for your project. You can read more about using Maven for this purpose on Maven.

Starting and stopping topologies on a remote cluster

The previous step installed the storm client on your machine which is used to communicate with remote Storm clusters. Now all you have to do is tell the client which Storm cluster to talk to. To do this, all you have to do is put the host address of the master in the ~/.storm/storm.yaml file. It should look something like this:

nimbus.host: "123.45.678.890"

Alternatively, if you use the storm-deploy project to provision Storm clusters on AWS, it will automatically set up your ~/.storm/storm.yaml file. You can manually attach to a Storm cluster (or switch between multiple clusters) using the "attach" command, like so:

lein run :deploy --attach --name mystormcluster

More information is on the storm-deploy wiki

storm环境搭建(前言)—— 翻译 Setting Up a Development Environment的更多相关文章

  1. storm环境搭建

    备注——使用: 1.单机版本: 启动zkServer.nimbus.supervisor.ui服务: zkServer.sh start zkServer.sh status #查看zkserver是 ...

  2. 【原】Storm环境搭建

    2.Storm环境搭建 单机 ... 集群 ... 搭建Storm开发环境 搭建Storm开发环境主要概括为以下两步: 1.下载Storm发行稳定版,然后解压,最后把解压后的bin/文件所在目录添加到 ...

  3. Storm环境搭建(分布式集群)

    作为流计算的开篇,笔者首先给出storm的安装和部署,storm的第二篇,笔者将详细的介绍storm的工作原理.下边直接上干货,跟笔者的步伐一块儿安装storm. 原文链接:Storm环境搭建(分布式 ...

  4. 170824、storm 环境搭建

    /*********************storm 环境搭建 **********************/ 1 安装jdk1.7 2 安装zookeeper集群环境 3 安装Python2.6. ...

  5. 《OD大数据实战》Storm环境搭建

    一.环境搭建 1. 下载 http://www.apache.org/dyn/closer.lua/storm/apache-storm-0.9.6/apache-storm-0.9.6.tar.gz ...

  6. 54.Storm环境搭建

    集群环境搭建 关闭防火墙,修改/etc/hosts配置(3台机器的ip可以相互通信) 下载安装jdk7(1.6以上),配置JAVA_HOME, CLASSPATH 搭建Zookeeper集群(保证3台 ...

  7. Jquery环境搭建前言

  8. apache - storm - Setting Up a Development Environment

    Installing a Storm release locally If you want to be able to submit topologies to a remote cluster f ...

  9. [Django] Setting up Django Development Environment in Ubuntu 14.04

    1. Python Of course you will need Python. Still Python 2.7 is preferred, however if you would like t ...

随机推荐

  1. [codeforces 293]B. Distinct Paths

    [codeforces 293]B. Distinct Paths 试题描述 You have a rectangular n × m-cell board. Some cells are alrea ...

  2. 一次手工注入waf [转载]

    转载自sss安全论坛 目标站点:http://www.xxx.cn:88注入点:http://www.xxx.cn:88/new/details1.asp?n_id=49909对其进行检测:http: ...

  3. RouterOS DNS劫持 -- A记录

    通常我们使用RouterOS的DNS主要是用于实现DNS缓存功能,即由RouterOS实现DNS服务器解析功能,除了这个功能,RouterOS可以实现对内网域名解析劫持,即实现路由网关的A记录查询. ...

  4. linux /etc/rc.d/目录的详解

    rc.d的内容如下: init.d/ :各种服务器和程序的二进制文件存放目录. rcx.d/: 各个启动级别的执行程序连接目录.里头的东西都是指向init.d/的一些软连接.具体的后边叙述. 还有三个 ...

  5. PHP网页数据正则采集

    $url ="https://********"; $contents = file_get_contents($url); //抓取页面数据 //如果出现中文乱码使用下面代码 / ...

  6. asp.net文本编辑器FCKeditor使用方法详解

    文本编辑器的使用: 1.FCKeditor的官方网站是:http://www.fckeditor.net/download  目前最新的FCKeditor.Net_2.6.9版本. 请在此页下载:ht ...

  7. 56. 2种方法判断二叉树是不是平衡二叉树[is balanced tree]

    [本文链接] http://www.cnblogs.com/hellogiser/p/is-balanced-tree.html [题目] 输入一棵二叉树的根结点,判断该树是不是平衡二叉树.如果某二叉 ...

  8. 39.递归颠倒栈[ReverseStack]

    [题目] 用递归颠倒一个栈.例如输入栈{1, 2, 3, 4, 5},1在栈顶.颠倒之后的栈为{5, 4, 3, 2, 1},5处在栈顶. [分析] 乍一看到这道题目,第一反应是把栈里的所有元素逐一p ...

  9. Cocos2d-x 学习资料推荐

    最近在看Cocos2d-x ,官网的资料太少了,下面推荐一些比较好的教程,不断更新中. 1. cocos2d-x高级开发教程 如果你懂得objective-c 那么一定要看看这本书,这里面有许多C++ ...

  10. cf_ducational Codeforces Round 16_D(gcd)

    题意:求R-L区间满足x=a1*k+b1=a2*l+b2的x的个数; 思路:求出最小的满足条件的x0,则ans=(L-x)/(a1/gcd(a1, a2)*a2)+1; 注意剪枝,不然会超时: 代码: ...