Storm编程入门API系列之Storm的Topology的stream grouping
概念，见博客
Storm概念学习系列之stream grouping（流分组）
Storm的stream grouping的Shuffle Grouping
它是随机分组，随机派发stream里面的tuple，保证polt的每个人物接收到的tuple数目相同。（它能实现较好的负载均衡）
如果工作中没有特殊要求，一般用Shuffle Grouping。
编写StormTopologyShufferGrouping.java
package zhouls.bigdata.stormDemo; import java.util.Map; import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseRichBolt;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils; /**
* shufferGrouping
* 没有特殊情况下，就使用这个分组方式，可以保证负载均衡，工作中最常用的
* @author zhouls
*
*/ public class StormTopologyShufferGrouping { public static class MySpout extends BaseRichSpout{
private Map conf;
private TopologyContext context;
private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context,
SpoutOutputCollector collector) {
this.conf = conf;
this.collector = collector;
this.context = context;
} int num = ; public void nextTuple() {
num++;
System.out.println("spout:"+num);
this.collector.emit(new Values(num));
Utils.sleep();
} public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("num"));
} } public static class MyBolt extends BaseRichBolt{ private Map stormConf;
private TopologyContext context;
private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.stormConf = stormConf;
this.context = context;
this.collector = collector;
} public void execute(Tuple input) {
Integer num = input.getIntegerByField("num");
System.err.println("thread:"+Thread.currentThread().getId()+",num="+num);
} public void declareOutputFields(OutputFieldsDeclarer declarer) { } } public static void main(String[] args) {
TopologyBuilder topologyBuilder = new TopologyBuilder();
String spout_id = MySpout.class.getSimpleName();
String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout());
topologyBuilder.setBolt(bolt_id, new MyBolt(),).shuffleGrouping(spout_id); Config config = new Config();
String topology_name = StormTopologyShufferGrouping.class.getSimpleName();
if(args.length==){
//在本地运行
LocalCluster localCluster = new LocalCluster();
localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology());
}else{
//在集群运行
try {
StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology());
} catch (AlreadyAliveException e) {
e.printStackTrace();
} catch (InvalidTopologyException e) {
e.printStackTrace();
} catch (AuthorizationException e) {
e.printStackTrace();
}
} } }
停掉，我们粘贴过来，分析分析
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3088ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2648ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1477ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2117ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1728ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2443ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3394ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.zookeeper - Queued up for leader lock.
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d86e269ef0001 type:create cxid:0x1 zxid:0x12 txntype:- reqpath:n/a Error Path:/storm/leader-lock Error:KeeperErrorCode = NoNode for /storm/leader-lock
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2891ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Curator-Framework-] WARN o.a.s.s.o.a.c.u.ZKPaths - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead.
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2881ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.d.m.MetricsUtils - Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
[main] INFO o.a.s.d.m.r.JmxPreparableReporter - Preparing...
[main] INFO o.a.s.d.common - Started statistics report plugin...
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@1b9d9a2b
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[timer] INFO o.a.s.d.nimbus - not a leader, skipping assignments
[timer] INFO o.a.s.d.nimbus - not a leader, skipping cleanup
[timer] INFO o.a.s.d.nimbus - not a leader skipping , credential renweal.
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2208ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2475ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef0005 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef0005, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[main-EventThread] INFO o.a.s.zookeeper - WIN-BQOBV63OBNM gained leadership, checking if it has all the topology code locally.
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86e269ef0005
[main-EventThread] INFO o.a.s.zookeeper - active-topology-ids [] local-topology-ids [] diff-topology []
[main-EventThread] INFO o.a.s.zookeeper - Accepting leadership, all active topology found localy.
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2785ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86e269ef0005 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86e269ef0005
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@fe87ddd
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@33eb6758
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3379ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef0006 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef0006, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1499ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef0007 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef0007, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86e269ef0007
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1938ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86e269ef0007
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86e269ef0007 closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@23da79eb
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2383ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef0008 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef0008, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\40e5b8bb-6b7f-422e-a1f6-63d7a0191a4b", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\40e5b8bb-6b7f-422e-a1f6-63d7a0191a4b\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\40e5b8bb-6b7f-422e-a1f6-63d7a0191a4b\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@642ee49c
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1275ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef0009, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef0009 with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86e269ef0009
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1887ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86e269ef0009
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86e269ef0009 closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@32456db0
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1014ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef000a, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef000a with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1060ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.d.supervisor - Starting supervisor with id 5a388ed1-6d49-41de-86be-4654e76a0525 at host WIN-BQOBV63OBNM
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\5f314744-35f9-428c-8f01-504459b2b6f4", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\5f314744-35f9-428c-8f01-504459b2b6f4\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\5f314744-35f9-428c-8f01-504459b2b6f4\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@29050de5
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1817ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef000b with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef000b, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1140ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86e269ef000b
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1733ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86e269ef000b
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86e269ef000b closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@71a3e05c
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2042ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef000c with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef000c, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1372ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.d.supervisor - Starting supervisor with id 8be87232--443f-a252-11532da04ae2 at host WIN-BQOBV63OBNM
[main] INFO o.a.s.l.ThriftAccessLogger - Request ID: access from: principal: operation: submitTopology
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1300ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.d.nimbus - Received topology submission for StormTopologyShufferGrouping with conf {"topology.max.task.parallelism" nil, "topology.submitter.principal" "", "topology.acker.executors" nil, "topology.eventlogger.executors" , "storm.zookeeper.superACL" nil, "topology.users" (), "topology.submitter.user" "Administrator", "topology.kryo.register" nil, "topology.kryo.decorators" (), "storm.id" "StormTopologyShufferGrouping-1-1501206655", "topology.name" "StormTopologyShufferGrouping"}
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1108ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1839ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1049ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.d.nimbus - uploadedJar
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5df63359
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef000d with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef000d, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2364ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d86e269ef000d type:create cxid:0x2 zxid:0x2b txntype:- reqpath:n/a Error Path:/storm/blobstoremaxkeysequencenumber Error:KeeperErrorCode = NoNode for /storm/blobstoremaxkeysequencenumber
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1631ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2340ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1663ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2149ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86e269ef000d
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1269ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2790ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86e269ef000d
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86e269ef000d closed
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyShufferGrouping---stormconf.ser/WIN-BQOBV63OBNM:-
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1204ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1086ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1727ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2643ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4dad0eed
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef000e with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef000e, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86e269ef000e
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86e269ef000e closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86e269ef000e
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyShufferGrouping---stormcode.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.d.nimbus - desired replication count achieved, current-replication-count for conf key = , current-replication-count for code key = , current-replication-count for jar key =
[main] INFO o.a.s.d.nimbus - Activating StormTopologyShufferGrouping: StormTopologyShufferGrouping--
[timer] INFO o.a.s.s.EvenScheduler - Available slots: (["8be87232-2246-443f-a252-11532da04ae2" ] ["8be87232-2246-443f-a252-11532da04ae2" ] ["8be87232-2246-443f-a252-11532da04ae2" ] ["5a388ed1-6d49-41de-86be-4654e76a0525" ] ["5a388ed1-6d49-41de-86be-4654e76a0525" ] ["5a388ed1-6d49-41de-86be-4654e76a0525" ])
[timer] INFO o.a.s.d.nimbus - Setting new assignment for topology id StormTopologyShufferGrouping--: #org.apache.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\06fd5cf4-ef41-4501-b9c6-09c306c2adf9", :node->host {"8be87232-2246-443f-a252-11532da04ae2" "WIN-BQOBV63OBNM"}, :executor->node+port {[ ] ["8be87232-2246-443f-a252-11532da04ae2" ], [ ] ["8be87232-2246-443f-a252-11532da04ae2" ], [ ] ["8be87232-2246-443f-a252-11532da04ae2" ], [ ] ["8be87232-2246-443f-a252-11532da04ae2" ], [ ] ["8be87232-2246-443f-a252-11532da04ae2" ]}, :executor->start-time-secs {[ ] , [ ] , [ ] , [ ] , [ ] }, :worker->resources {["8be87232-2246-443f-a252-11532da04ae2" ] [0.0 0.0 0.0]}}
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyShufferGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@6ad941a5
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\06fd5cf4-ef41--b9c6-09c306c2adf9\blobs
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef000f with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef000f, negotiated timeout =
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86e269ef000f
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1497ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86e269ef000f
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86e269ef000f closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.d.supervisor - Removing code for storm id StormTopologyShufferGrouping--
[Thread-] INFO o.a.s.d.supervisor - java.io.FileNotFoundException: File 'C:\Users\ADMINI~1\AppData\Local\Temp\5f314744-35f9-428c-8f01-504459b2b6f4\supervisor\stormdist\StormTopologyShufferGrouping-1-1501206655\stormconf.ser' does not existException removing: StormTopologyShufferGrouping--
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2752ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyShufferGrouping--
[Thread-] INFO o.a.s.d.supervisor - Launching worker with assignment {:storm-id "StormTopologyShufferGrouping-1-1501206655", :executors [[ ] [ ] [ ] [ ] [ ]], :resources #object[org.apache.storm.generated.WorkerResources 0x5e199f91 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor 8be87232--443f-a252-11532da04ae2 on port with id 827c72b2-aabc--87d3-809999df55ff
[Thread-] INFO o.a.s.d.worker - Launching worker for StormTopologyShufferGrouping-- on 8be87232--443f-a252-11532da04ae2: with id 827c72b2-aabc--87d3-809999df55ff and conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\5f314744-35f9-428c-8f01-504459b2b6f4", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5435521d
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1782ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2264ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef0010 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef0010, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread--EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86e269ef0010
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2052ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86e269ef0010 closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@65132f3f
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d86e269ef0010, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86e269ef0010
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3081ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3074ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86e269ef0011 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86e269ef0011, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2787ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3209ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.s.a.AuthUtils - Got AutoCreds []
[Thread-] INFO o.a.s.d.worker - Reading Assignments.
[Thread-] INFO o.a.s.d.worker - Registering IConnectionCallbacks for 8be87232--443f-a252-11532da04ae2:
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[refresh-active-timer] INFO o.a.s.d.worker - All connections are ready for worker 8be87232--443f-a252-11532da04ae2: with id 827c72b2-aabc--87d3-809999df55ff
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __system:[- -]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Timeouts disabled for executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __acker:[ ]
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2971ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.d.executor - Loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.worker - Started with log levels: {"" #object[org.apache.logging.log4j.Level 0x53b3a9bb "INFO"], "org.apache.zookeeper" #object[org.apache.logging.log4j.Level 0x6be709b5 "WARN"]}
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2755ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3833ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.d.worker - Worker has topology config {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "topology.submitter.principal" "", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\06fd5cf4-ef41-4501-b9c6-09c306c2adf9", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "storm.zookeeper.superACL" nil, "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" [ ], "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "topology.users" [], "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.submitter.user" "Administrator", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "topology.kryo.register" nil, "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "topology.kryo.decorators" [], "storm.id" "StormTopologyShufferGrouping-1-1501206655", "topology.name" "StormTopologyShufferGrouping", "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.d.worker - Worker 827c72b2-aabc--87d3-809999df55ff for storm StormTopologyShufferGrouping-- on 8be87232--443f-a252-11532da04ae2: has finished loading
[Thread-] INFO o.a.s.config - SET worker-user 827c72b2-aabc--87d3-809999df55ff
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3428ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2039ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Preparing bolt __system:(-)
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opening spout MySpout:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opened spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Activating spout MySpout:()
spout:
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Prepared bolt __system:(-)
thread:,num=
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Preparing bolt __acker:()
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Prepared bolt __acker:()
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 5001ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3994ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2892ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3336ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3051ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
我们可以看到，总共是3个线程。
thread:122 、thread:124、thread:126
Storm的stream grouping的Fields Grouping
它是按字段分组，比如按userid来分组，具有同样useid的tuple会被分到同一任务，而不同的userid则会被分配到不同的任务。
编写代码StormTopologyFieldsGrouping.java
package zhouls.bigdata.stormDemo; import java.util.Map; import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseRichBolt;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils; /**
* FieldsGrouping
* 字段分组
* @author zhouls
*
*/ public class StormTopologyFieldsGrouping { public static class MySpout extends BaseRichSpout{
private Map conf;
private TopologyContext context;
private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context,
SpoutOutputCollector collector) {
this.conf = conf;
this.collector = collector;
this.context = context;
} int num = ; public void nextTuple() {
num++;
System.out.println("spout:"+num);
this.collector.emit(new Values(num,num%));
Utils.sleep();
} public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("num","flag"));
} } public static class MyBolt extends BaseRichBolt{ private Map stormConf;
private TopologyContext context;
private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.stormConf = stormConf;
this.context = context;
this.collector = collector;
} public void execute(Tuple input) {
Integer num = input.getIntegerByField("num");
System.err.println("thread:"+Thread.currentThread().getId()+",num="+num);
} public void declareOutputFields(OutputFieldsDeclarer declarer) { } } public static void main(String[] args) {
TopologyBuilder topologyBuilder = new TopologyBuilder();
String spout_id = MySpout.class.getSimpleName();
String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout());
//注意：字段分组一定可以保证相同分组的数据进入同一个线程处理
topologyBuilder.setBolt(bolt_id, new MyBolt(),).fieldsGrouping(spout_id, new Fields("flag"));
//这个Fields字段从spolt里来 Config config = new Config();
String topology_name = StormTopologyFieldsGrouping.class.getSimpleName();
if(args.length==){
//在本地运行
LocalCluster localCluster = new LocalCluster();
localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology());
}else{
//在集群运行
try {
StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology());
} catch (AlreadyAliveException e) {
e.printStackTrace();
} catch (InvalidTopologyException e) {
e.printStackTrace();
} catch (AuthorizationException e) {
e.printStackTrace();
}
} } }
停掉之后，我们来粘贴分析分析
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0000 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0000, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0001 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0001, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5b5dce5c
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0002 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0002, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be0002
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be0002 closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@250a9031
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be0002
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@70025b99
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0003 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0003, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0004 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0004, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.zookeeper - Queued up for leader lock.
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d86eee5be0001 type:create cxid:0x1 zxid:0x12 txntype:- reqpath:n/a Error Path:/storm/leader-lock Error:KeeperErrorCode = NoNode for /storm/leader-lock
[Curator-Framework-] WARN o.a.s.s.o.a.c.u.ZKPaths - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead.
[main] INFO o.a.s.d.m.MetricsUtils - Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
[main] INFO o.a.s.d.m.r.JmxPreparableReporter - Preparing...
[timer] INFO o.a.s.d.nimbus - not a leader, skipping assignments
[timer] INFO o.a.s.d.nimbus - not a leader, skipping cleanup
[timer] INFO o.a.s.d.nimbus - not a leader skipping , credential renweal.
[main-EventThread] INFO o.a.s.zookeeper - WIN-BQOBV63OBNM gained leadership, checking if it has all the topology code locally.
[main] INFO o.a.s.d.common - Started statistics report plugin...
[main-EventThread] INFO o.a.s.zookeeper - active-topology-ids [] local-topology-ids [] diff-topology []
[main-EventThread] INFO o.a.s.zookeeper - Accepting leadership, all active topology found localy.
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@34d713a2
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0005, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0005 with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be0005
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be0005 closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be0005
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@60aec68a
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@17dad32f
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0006, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0006 with negotiated timeout for client /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0007 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0007, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be0007
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be0007 closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be0007
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@7c281eb8
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0008 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0008, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\80c9936b-abea-4b55-b5e1-5aef69139b32", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\80c9936b-abea-4b55-b5e1-5aef69139b32\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\80c9936b-abea-4b55-b5e1-5aef69139b32\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@de81be1
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0009 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0009, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be0009
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be0009
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be0009 closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@7237f3c1
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be000a with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be000a, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id 4fee4190--4bb8-9b33-1b791d3aaab9 at host WIN-BQOBV63OBNM
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\2be48e75-c7d5-4d64-87b5-62c8d231e67f", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\2be48e75-c7d5-4d64-87b5-62c8d231e67f\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\2be48e75-c7d5-4d64-87b5-62c8d231e67f\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@78fe204a
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be000b with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be000b, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be000b
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be000b closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4091b9c3
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be000b
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be000c with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be000c, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id ac253c5a-a5db---ef70ab900b87 at host WIN-BQOBV63OBNM
[main] INFO o.a.s.l.ThriftAccessLogger - Request ID: access from: principal: operation: submitTopology
[main] INFO o.a.s.d.nimbus - Received topology submission for StormTopologyFieldsGrouping with conf {"topology.max.task.parallelism" nil, "topology.submitter.principal" "", "topology.acker.executors" nil, "topology.eventlogger.executors" , "storm.zookeeper.superACL" nil, "topology.users" (), "topology.submitter.user" "Administrator", "topology.kryo.register" nil, "topology.kryo.decorators" (), "storm.id" "StormTopologyFieldsGrouping-1-1501207396", "topology.name" "StormTopologyFieldsGrouping"}
[main] INFO o.a.s.d.nimbus - uploadedJar
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5b39a3e6
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be000d, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be000d with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d86eee5be000d type:create cxid:0x2 zxid:0x27 txntype:- reqpath:n/a Error Path:/storm/blobstoremaxkeysequencenumber Error:KeeperErrorCode = NoNode for /storm/blobstoremaxkeysequencenumber
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be000d
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be000d closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be000d
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyFieldsGrouping---stormconf.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@75cf0de5
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be000e with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be000e, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be000e
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be000e closed
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyFieldsGrouping---stormcode.ser/WIN-BQOBV63OBNM:-
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d86eee5be000e, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be000e
[main] INFO o.a.s.d.nimbus - desired replication count achieved, current-replication-count for conf key = , current-replication-count for code key = , current-replication-count for jar key =
[main] INFO o.a.s.d.nimbus - Activating StormTopologyFieldsGrouping: StormTopologyFieldsGrouping--
[timer] INFO o.a.s.s.EvenScheduler - Available slots: (["ac253c5a-a5db-4266-9292-ef70ab900b87" ] ["ac253c5a-a5db-4266-9292-ef70ab900b87" ] ["ac253c5a-a5db-4266-9292-ef70ab900b87" ] ["4fee4190-1280-4bb8-9b33-1b791d3aaab9" ] ["4fee4190-1280-4bb8-9b33-1b791d3aaab9" ] ["4fee4190-1280-4bb8-9b33-1b791d3aaab9" ])
[timer] INFO o.a.s.d.nimbus - Setting new assignment for topology id StormTopologyFieldsGrouping--: #org.apache.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\04e95801-7c28-4111-8653-9f075dd17057", :node->host {"ac253c5a-a5db-4266-9292-ef70ab900b87" "WIN-BQOBV63OBNM"}, :executor->node+port {[ ] ["ac253c5a-a5db-4266-9292-ef70ab900b87" ], [ ] ["ac253c5a-a5db-4266-9292-ef70ab900b87" ], [ ] ["ac253c5a-a5db-4266-9292-ef70ab900b87" ], [ ] ["ac253c5a-a5db-4266-9292-ef70ab900b87" ]}, :executor->start-time-secs {[ ] , [ ] , [ ] , [ ] }, :worker->resources {["ac253c5a-a5db-4266-9292-ef70ab900b87" ] [0.0 0.0 0.0]}}
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@715bd16b
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\04e95801-7c28---9f075dd17057\blobs
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be000f with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be000f, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be000f
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be000f closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d86eee5be000f, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be000f
[Thread-] INFO o.a.s.d.supervisor - Removing code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyFieldsGrouping-1-1501207396", :executors [[4 4] [3 3] [2 2] [1 1]], :resources #object[org.apache.storm.generated.WorkerResources 0x36d91263 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ac253c5a-a5db-4266-9292-ef70ab900b87 on port 1027 with id 6cc990a9-a735-43c0-9ca5-62d8564b5c62
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@a0899fa
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\04e95801-7c28---9f075dd17057\blobs
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0010 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0010, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be0010
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be0010 closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be0010
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.d.supervisor - Launching worker with assignment {:storm-id "StormTopologyFieldsGrouping-1-1501207396", :executors [[ ] [ ] [ ] [ ]], :resources #object[org.apache.storm.generated.WorkerResources 0x7f2e317f "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ac253c5a-a5db---ef70ab900b87 on port with id a91d003f-d6cf-4d14-8d87-22da12c85bb5
[Thread-] INFO o.a.s.d.worker - Launching worker for StormTopologyFieldsGrouping-- on ac253c5a-a5db---ef70ab900b87: with id a91d003f-d6cf-4d14-8d87-22da12c85bb5 and conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\2be48e75-c7d5-4d64-87b5-62c8d231e67f", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@2de9221a
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0011 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0011, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread--EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86eee5be0011
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86eee5be0011 closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d86eee5be0011, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86eee5be0011
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@440b362b
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86eee5be0012, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86eee5be0012 with negotiated timeout for client /127.0.0.1:
[Thread-] INFO o.a.s.s.a.AuthUtils - Got AutoCreds []
[Thread-] INFO o.a.s.d.worker - Reading Assignments.
[Thread-] INFO o.a.s.d.worker - Registering IConnectionCallbacks for ac253c5a-a5db---ef70ab900b87:
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[refresh-active-timer] INFO o.a.s.d.worker - All connections are ready for worker ac253c5a-a5db---ef70ab900b87: with id a91d003f-d6cf-4d14-8d87-22da12c85bb5
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __system:[- -]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Timeouts disabled for executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.worker - Started with log levels: {"" #object[org.apache.logging.log4j.Level 0x364b182b "INFO"], "org.apache.zookeeper" #object[org.apache.logging.log4j.Level 0x268b8ab3 "WARN"]}
[Thread-] INFO o.a.s.d.worker - Worker has topology config {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "topology.submitter.principal" "", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\04e95801-7c28-4111-8653-9f075dd17057", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "storm.zookeeper.superACL" nil, "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" [ ], "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "topology.users" [], "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.submitter.user" "Administrator", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "topology.kryo.register" nil, "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "topology.kryo.decorators" [], "storm.id" "StormTopologyFieldsGrouping-1-1501207396", "topology.name" "StormTopologyFieldsGrouping", "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.d.worker - Worker a91d003f-d6cf-4d14-8d87-22da12c85bb5 for storm StormTopologyFieldsGrouping-- on ac253c5a-a5db---ef70ab900b87: has finished loading
[Thread-] INFO o.a.s.config - SET worker-user a91d003f-d6cf-4d14-8d87-22da12c85bb5
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Preparing bolt __system:(-)
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Preparing bolt __acker:()
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Prepared bolt __acker:()
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Prepared bolt __system:(-)
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opening spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opened spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Activating spout MySpout:()
spout:
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1731ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1371ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1771ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1041ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
可以看出，thread:126全为偶数。thread:130全为奇数。
为什么是两个线程。是因为
topologyBuilder.setBolt(bolt_id, new MyBolt(),).fieldsGrouping(spout_id, new Fields("flag"));
即，以后大家可以在我这代码上修改拿去用。字段分组。
比如大家有6个，则对应修改就是
topologyBuilder.setBolt(bolt_id, new MyBolt(),).fieldsGrouping(spout_id, new Fields("*****"));
多个字段，对应分到对应的线程去跑。
Storm的stream grouping的All Grouping
它是广播发送，对于每一个tuple，Bolts中的所有任务都会收到。
即多个线程就重复了嘛
编写代码StormTopologyAllGrouping.java
package zhouls.bigdata.stormDemo; import java.util.Map; import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseRichBolt;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils; /**
* AllGrouping
* 广播分组
* @author zhouls
*
*/ public class StormTopologyAllGrouping { public static class MySpout extends BaseRichSpout{
private Map conf;
private TopologyContext context;
private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context,
SpoutOutputCollector collector) {
this.conf = conf;
this.collector = collector;
this.context = context;
} int num = ; public void nextTuple() {
num++;
System.out.println("spout:"+num);
this.collector.emit(new Values(num));
Utils.sleep();
} public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("num"));
} } public static class MyBolt extends BaseRichBolt{ private Map stormConf;
private TopologyContext context;
private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.stormConf = stormConf;
this.context = context;
this.collector = collector;
} public void execute(Tuple input) {
Integer num = input.getIntegerByField("num");
System.err.println("thread:"+Thread.currentThread().getId()+",num="+num);
} public void declareOutputFields(OutputFieldsDeclarer declarer) { } } public static void main(String[] args) {
TopologyBuilder topologyBuilder = new TopologyBuilder();
String spout_id = MySpout.class.getSimpleName();
String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout());
topologyBuilder.setBolt(bolt_id, new MyBolt(),).allGrouping(spout_id); Config config = new Config();
String topology_name = StormTopologyAllGrouping.class.getSimpleName();
if(args.length==){
//在本地运行
LocalCluster localCluster = new LocalCluster();
localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology());
}else{
//在集群运行
try {
StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology());
} catch (AlreadyAliveException e) {
e.printStackTrace();
} catch (InvalidTopologyException e) {
e.printStackTrace();
} catch (AuthorizationException e) {
e.printStackTrace();
}
} } }
停掉，我们来粘贴分析分析
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3134ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330000 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330000, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1787ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330001 with negotiated timeout for client /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330002 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330001, negotiated timeout =
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330002, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2666ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f840330002
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3159ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f840330002
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f840330002 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@40de8f93
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@45ab3bdd
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2239ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330003 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330003, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1857ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330004 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330004, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1197ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1888ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1847ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.zookeeper - Queued up for leader lock.
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d86f840330001 type:create cxid:0x1 zxid:0x12 txntype:- reqpath:n/a Error Path:/storm/leader-lock Error:KeeperErrorCode = NoNode for /storm/leader-lock
[Curator-Framework-] WARN o.a.s.s.o.a.c.u.ZKPaths - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead.
[main] INFO o.a.s.d.m.MetricsUtils - Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
[main] INFO o.a.s.d.m.r.JmxPreparableReporter - Preparing...
[main-EventThread] INFO o.a.s.zookeeper - WIN-BQOBV63OBNM gained leadership, checking if it has all the topology code locally.
[main] INFO o.a.s.d.common - Started statistics report plugin...
[main-EventThread] INFO o.a.s.zookeeper - active-topology-ids [] local-topology-ids [] diff-topology []
[main-EventThread] INFO o.a.s.zookeeper - Accepting leadership, all active topology found localy.
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@17dad32f
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330005 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330005, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f840330005
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f840330005
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f840330005 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@7c281eb8
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4a8ffd75
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330006 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330006, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330007 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330007, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f840330007
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f840330007
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f840330007 closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3e05586b
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330008 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330008, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\05ef2d4d-38b4-48eb-a29d-45ca1a9f21cf", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\05ef2d4d-38b4-48eb-a29d-45ca1a9f21cf\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\05ef2d4d-38b4-48eb-a29d-45ca1a9f21cf\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@294aba23
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330009 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330009, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f840330009
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f840330009
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f840330009 closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5f5827d0
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f84033000a, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f84033000a with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id 65e835bf-c14e--9f66-5345516d7f1f at host WIN-BQOBV63OBNM
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\fd0d1b90-b12e-4ab8-ba25-bb46f39442ee", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\fd0d1b90-b12e-4ab8-ba25-bb46f39442ee\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\fd0d1b90-b12e-4ab8-ba25-bb46f39442ee\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@c689973
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f84033000b with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f84033000b, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f84033000b
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f84033000b
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f84033000b closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@2b148329
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f84033000c with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f84033000c, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id cd418233-288a-4fe9-8f9d-6471907095a8 at host WIN-BQOBV63OBNM
[main] INFO o.a.s.l.ThriftAccessLogger - Request ID: access from: principal: operation: submitTopology
[main] INFO o.a.s.d.nimbus - Received topology submission for StormTopologyAllGrouping with conf {"topology.max.task.parallelism" nil, "topology.submitter.principal" "", "topology.acker.executors" nil, "topology.eventlogger.executors" , "storm.zookeeper.superACL" nil, "topology.users" (), "topology.submitter.user" "Administrator", "topology.kryo.register" nil, "topology.kryo.decorators" (), "storm.id" "StormTopologyAllGrouping-1-1501208026", "topology.name" "StormTopologyAllGrouping"}
[main] INFO o.a.s.d.nimbus - uploadedJar
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@460b50df
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f84033000d with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f84033000d, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d86f84033000d type:create cxid:0x2 zxid:0x27 txntype:- reqpath:n/a Error Path:/storm/blobstoremaxkeysequencenumber Error:KeeperErrorCode = NoNode for /storm/blobstoremaxkeysequencenumber
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f84033000d
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f84033000d
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f84033000d closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyAllGrouping---stormconf.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@2cd388f5
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f84033000e with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f84033000e, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f84033000e
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f84033000e closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f84033000e
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyAllGrouping---stormcode.ser/WIN-BQOBV63OBNM:-
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1075ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.d.nimbus - desired replication count achieved, current-replication-count for conf key = , current-replication-count for code key = , current-replication-count for jar key =
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3826ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1513ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.d.nimbus - Activating StormTopologyAllGrouping: StormTopologyAllGrouping--
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1884ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[timer] INFO o.a.s.s.EvenScheduler - Available slots: (["65e835bf-c14e-4824-9f66-5345516d7f1f" ] ["65e835bf-c14e-4824-9f66-5345516d7f1f" ] ["65e835bf-c14e-4824-9f66-5345516d7f1f" ] ["cd418233-288a-4fe9-8f9d-6471907095a8" ] ["cd418233-288a-4fe9-8f9d-6471907095a8" ] ["cd418233-288a-4fe9-8f9d-6471907095a8" ])
[timer] INFO o.a.s.d.nimbus - Setting new assignment for topology id StormTopologyAllGrouping--: #org.apache.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\15fb7541-e2b3-47cc-96a9-028aa7736abd", :node->host {"65e835bf-c14e-4824-9f66-5345516d7f1f" "WIN-BQOBV63OBNM"}, :executor->node+port {[ ] ["65e835bf-c14e-4824-9f66-5345516d7f1f" ], [ ] ["65e835bf-c14e-4824-9f66-5345516d7f1f" ], [ ] ["65e835bf-c14e-4824-9f66-5345516d7f1f" ], [ ] ["65e835bf-c14e-4824-9f66-5345516d7f1f" ], [ ] ["65e835bf-c14e-4824-9f66-5345516d7f1f" ]}, :executor->start-time-secs {[ ] , [ ] , [ ] , [ ] , [ ] }, :worker->resources {["65e835bf-c14e-4824-9f66-5345516d7f1f" ] [0.0 0.0 0.0]}}
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3479ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2780ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyAllGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3dc76165
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\15fb7541-e2b3-47cc-96a9-028aa7736abd\blobs
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3710ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f84033000f with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f84033000f, negotiated timeout =
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f84033000f
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2192ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f84033000f
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f84033000f closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.d.supervisor - Removing code for storm id StormTopologyAllGrouping--
[Thread-] INFO o.a.s.d.supervisor - java.io.FileNotFoundException: File 'C:\Users\ADMINI~1\AppData\Local\Temp\05ef2d4d-38b4-48eb-a29d-45ca1a9f21cf\supervisor\stormdist\StormTopologyAllGrouping-1-1501208026\stormconf.ser' does not existException removing: StormTopologyAllGrouping--
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2936ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyAllGrouping--
[Thread-] INFO o.a.s.d.supervisor - Launching worker with assignment {:storm-id "StormTopologyAllGrouping-1-1501208026", :executors [[ ] [ ] [ ] [ ] [ ]], :resources #object[org.apache.storm.generated.WorkerResources 0x2ab0a801 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor 65e835bf-c14e--9f66-5345516d7f1f on port with id 1f0b1c16-1f76-41d6-a4cf-03d1f528366a
[Thread-] INFO o.a.s.d.worker - Launching worker for StormTopologyAllGrouping-- on 65e835bf-c14e--9f66-5345516d7f1f: with id 1f0b1c16-1f76-41d6-a4cf-03d1f528366a and conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\05ef2d4d-38b4-48eb-a29d-45ca1a9f21cf", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@71aefd31
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3111ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2328ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330010 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330010, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread--EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d86f840330010
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2379ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2203ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d86f840330010 closed
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@268d7dbf
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d86f840330010
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2517ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2381ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d86f840330011 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d86f840330011, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1520ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.s.a.AuthUtils - Got AutoCreds []
[Thread-] INFO o.a.s.d.worker - Reading Assignments.
[Thread-] INFO o.a.s.d.worker - Registering IConnectionCallbacks for 65e835bf-c14e--9f66-5345516d7f1f:
[refresh-active-timer] INFO o.a.s.d.worker - All connections are ready for worker 65e835bf-c14e--9f66-5345516d7f1f: with id 1f0b1c16-1f76-41d6-a4cf-03d1f528366a
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1423ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __system:[- -]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Timeouts disabled for executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.worker - Started with log levels: {"" #object[org.apache.logging.log4j.Level 0x33b220b0 "INFO"], "org.apache.zookeeper" #object[org.apache.logging.log4j.Level 0x6e0970c0 "WARN"]}
[Thread-] INFO o.a.s.d.worker - Worker has topology config {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "topology.submitter.principal" "", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\15fb7541-e2b3-47cc-96a9-028aa7736abd", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "storm.zookeeper.superACL" nil, "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" [ ], "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "topology.users" [], "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.submitter.user" "Administrator", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "topology.kryo.register" nil, "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "topology.kryo.decorators" [], "storm.id" "StormTopologyAllGrouping-1-1501208026", "topology.name" "StormTopologyAllGrouping", "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.d.worker - Worker 1f0b1c16-1f76-41d6-a4cf-03d1f528366a for storm StormTopologyAllGrouping-- on 65e835bf-c14e--9f66-5345516d7f1f: has finished loading
[Thread-] INFO o.a.s.config - SET worker-user 1f0b1c16-1f76-41d6-a4cf-03d1f528366a
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Preparing bolt __acker:()
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Prepared bolt __acker:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Preparing bolt __system:(-)
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Prepared bolt __system:(-)
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opening spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opened spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Activating spout MySpout:()
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2528ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
thread:,num=
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1155ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
thread:,num=
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1226ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1071ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
spout:
thread:,num=
thread:,num=
thread:,num=
即spolt每发，进程都会接收到。
至于，为什么是3个进程。是因为
topologyBuilder.setBolt(bolt_id, new MyBolt(),).allGrouping(spout_id);
Storm的stream grouping的Global Grouping
它是全局分组，这个tuple被分配到storm中的一个bolt的其中一个task。再具体一点就是分配给id值最低的那个task。
编写代码StormTopologyGlobalGrouping.java
package zhouls.bigdata.stormDemo; import java.util.Map; import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseRichBolt;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils; /**
* GlobalGrouping
* 全局分组
* @author zhouls
*
*/ public class StormTopologyGlobalGrouping { public static class MySpout extends BaseRichSpout{
private Map conf;
private TopologyContext context;
private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context,
SpoutOutputCollector collector) {
this.conf = conf;
this.collector = collector;
this.context = context;
} int num = ; public void nextTuple() {
num++;
System.out.println("spout:"+num);
this.collector.emit(new Values(num,num%));
Utils.sleep();
} public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("num","flag"));
} } public static class MyBolt extends BaseRichBolt{ private Map stormConf;
private TopologyContext context;
private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.stormConf = stormConf;
this.context = context;
this.collector = collector;
} public void execute(Tuple input) {
Integer num = input.getIntegerByField("num");
System.err.println("thread:"+Thread.currentThread().getId()+",num="+num);
} public void declareOutputFields(OutputFieldsDeclarer declarer) { } } public static void main(String[] args) {
TopologyBuilder topologyBuilder = new TopologyBuilder();
String spout_id = MySpout.class.getSimpleName();
String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout());
topologyBuilder.setBolt(bolt_id, new MyBolt(),).globalGrouping(spout_id); Config config = new Config();
String topology_name = StormTopologyFieldsGrouping.class.getSimpleName();
if(args.length==){
//在本地运行
LocalCluster localCluster = new LocalCluster();
localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology());
}else{
//在集群运行
try {
StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology());
} catch (AlreadyAliveException e) {
e.printStackTrace();
} catch (InvalidTopologyException e) {
e.printStackTrace();
} catch (AuthorizationException e) {
e.printStackTrace();
}
} } }
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5b5dce5c
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520000 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520000, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520001 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520001, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520002 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520002, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f520002
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f520002 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f520002
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3e1fd62b
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@40de8f93
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520003 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520003, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520004 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520004, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.zookeeper - Queued up for leader lock.
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d87157f520001 type:create cxid:0x1 zxid:0x12 txntype:- reqpath:n/a Error Path:/storm/leader-lock Error:KeeperErrorCode = NoNode for /storm/leader-lock
[Curator-Framework-] WARN o.a.s.s.o.a.c.u.ZKPaths - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead.
[main] INFO o.a.s.d.m.MetricsUtils - Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
[main] INFO o.a.s.d.m.r.JmxPreparableReporter - Preparing...
[timer] INFO o.a.s.d.nimbus - not a leader, skipping assignments
[timer] INFO o.a.s.d.nimbus - not a leader, skipping cleanup
[timer] INFO o.a.s.d.nimbus - not a leader skipping , credential renweal.
[main] INFO o.a.s.d.common - Started statistics report plugin...
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@36f7d7b
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-EventThread] INFO o.a.s.zookeeper - WIN-BQOBV63OBNM gained leadership, checking if it has all the topology code locally.
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520005 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520005, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f520005
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f520005 closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@d919544
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d87157f520005, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4eea94a4
[main-EventThread] INFO o.a.s.zookeeper - active-topology-ids [] local-topology-ids [] diff-topology []
[main-EventThread] INFO o.a.s.zookeeper - Accepting leadership, all active topology found localy.
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f520005
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520006 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520006, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520007, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520007 with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f520007
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f520007
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f520007 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@f8a6243
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520008 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520008, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\b1e382b7-e58c-40aa-915d-0e8a630a2184", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\b1e382b7-e58c-40aa-915d-0e8a630a2184\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\b1e382b7-e58c-40aa-915d-0e8a630a2184\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3596b249
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520009 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520009, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f520009
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f520009 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f520009
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@294aba23
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f52000a with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f52000a, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id a833b5aa-1e2f--b03f-92e032688c0e at host WIN-BQOBV63OBNM
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\e985592b-f57a-43a5-b7ec-2854e9db2d2b", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\e985592b-f57a-43a5-b7ec-2854e9db2d2b\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\e985592b-f57a-43a5-b7ec-2854e9db2d2b\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@55fdf7f9
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f52000b with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f52000b, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f52000b
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f52000b
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f52000b closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@41fa769c
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f52000c with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f52000c, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id ed6a51e1-c479-48f2-b386-7d0278d524a4 at host WIN-BQOBV63OBNM
[main] INFO o.a.s.l.ThriftAccessLogger - Request ID: access from: principal: operation: submitTopology
[main] INFO o.a.s.d.nimbus - Received topology submission for StormTopologyFieldsGrouping with conf {"topology.max.task.parallelism" nil, "topology.submitter.principal" "", "topology.acker.executors" nil, "topology.eventlogger.executors" , "storm.zookeeper.superACL" nil, "topology.users" (), "topology.submitter.user" "Administrator", "topology.kryo.register" nil, "topology.kryo.decorators" (), "storm.id" "StormTopologyFieldsGrouping-1-1501209922", "topology.name" "StormTopologyFieldsGrouping"}
[main] INFO o.a.s.d.nimbus - uploadedJar
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@67d32a54
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f52000d with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f52000d, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d87157f52000d type:create cxid:0x2 zxid:0x27 txntype:- reqpath:n/a Error Path:/storm/blobstoremaxkeysequencenumber Error:KeeperErrorCode = NoNode for /storm/blobstoremaxkeysequencenumber
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f52000d
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f52000d
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f52000d closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyFieldsGrouping---stormconf.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@57efc6fd
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f52000e with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f52000e, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f52000e
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f52000e
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f52000e closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyFieldsGrouping---stormcode.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.d.nimbus - desired replication count achieved, current-replication-count for conf key = , current-replication-count for code key = , current-replication-count for jar key =
[main] INFO o.a.s.d.nimbus - Activating StormTopologyFieldsGrouping: StormTopologyFieldsGrouping--
[timer] INFO o.a.s.s.EvenScheduler - Available slots: (["ed6a51e1-c479-48f2-b386-7d0278d524a4" ] ["ed6a51e1-c479-48f2-b386-7d0278d524a4" ] ["ed6a51e1-c479-48f2-b386-7d0278d524a4" ] ["a833b5aa-1e2f-4819-b03f-92e032688c0e" ] ["a833b5aa-1e2f-4819-b03f-92e032688c0e" ] ["a833b5aa-1e2f-4819-b03f-92e032688c0e" ])
[timer] INFO o.a.s.d.nimbus - Setting new assignment for topology id StormTopologyFieldsGrouping--: #org.apache.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\84404381-9d44-4573-a9cb-12e6e4b48d4c", :node->host {"ed6a51e1-c479-48f2-b386-7d0278d524a4" "WIN-BQOBV63OBNM"}, :executor->node+port {[ ] ["ed6a51e1-c479-48f2-b386-7d0278d524a4" ], [ ] ["ed6a51e1-c479-48f2-b386-7d0278d524a4" ], [ ] ["ed6a51e1-c479-48f2-b386-7d0278d524a4" ], [ ] ["ed6a51e1-c479-48f2-b386-7d0278d524a4" ]}, :executor->start-time-secs {[ ] , [ ] , [ ] , [ ] }, :worker->resources {["ed6a51e1-c479-48f2-b386-7d0278d524a4" ] [0.0 0.0 0.0]}}
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@382a8558
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\-9d44--a9cb-12e6e4b48d4c\blobs
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f52000f with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f52000f, negotiated timeout =
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f52000f
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f52000f closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f52000f
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.d.supervisor - Removing code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyFieldsGrouping-1-1501209922", :executors [[4 4] [3 3] [2 2] [1 1]], :resources #object[org.apache.storm.generated.WorkerResources 0x1fd11f75 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ed6a51e1-c479-48f2-b386-7d0278d524a4 on port 1027 with id b4268638-a841-439b-b58a-9366d6b91226
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@67b3bb1b
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\-9d44--a9cb-12e6e4b48d4c\blobs
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520010 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520010, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f520010
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d87157f520010, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f520010
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f520010 closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.d.supervisor - Launching worker with assignment {:storm-id "StormTopologyFieldsGrouping-1-1501209922", :executors [[ ] [ ] [ ] [ ]], :resources #object[org.apache.storm.generated.WorkerResources 0x3e290c0e "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ed6a51e1-c479-48f2-b386-7d0278d524a4 on port with id 95ac2248--43a5-8daf-03818ac27a03
[Thread-] INFO o.a.s.d.worker - Launching worker for StormTopologyFieldsGrouping-- on ed6a51e1-c479-48f2-b386-7d0278d524a4: with id 95ac2248--43a5-8daf-03818ac27a03 and conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\e985592b-f57a-43a5-b7ec-2854e9db2d2b", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@36551e11
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520011, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520011 with negotiated timeout for client /127.0.0.1:
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread--EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87157f520011
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87157f520011 closed
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87157f520011
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@640f9cdc
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87157f520012 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87157f520012, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread-] INFO o.a.s.s.a.AuthUtils - Got AutoCreds []
[Thread-] INFO o.a.s.d.worker - Reading Assignments.
[Thread-] INFO o.a.s.d.worker - Registering IConnectionCallbacks for ed6a51e1-c479-48f2-b386-7d0278d524a4:
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[refresh-active-timer] INFO o.a.s.d.worker - All connections are ready for worker ed6a51e1-c479-48f2-b386-7d0278d524a4: with id 95ac2248--43a5-8daf-03818ac27a03
[Thread-] INFO o.a.s.d.executor - Loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __system:[- -]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Timeouts disabled for executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.worker - Started with log levels: {"" #object[org.apache.logging.log4j.Level 0x31665167 "INFO"], "org.apache.zookeeper" #object[org.apache.logging.log4j.Level 0x60661ce8 "WARN"]}
[Thread-] INFO o.a.s.d.worker - Worker has topology config {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "topology.submitter.principal" "", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\84404381-9d44-4573-a9cb-12e6e4b48d4c", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "storm.zookeeper.superACL" nil, "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" [ ], "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "topology.users" [], "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.submitter.user" "Administrator", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "topology.kryo.register" nil, "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "topology.kryo.decorators" [], "storm.id" "StormTopologyFieldsGrouping-1-1501209922", "topology.name" "StormTopologyFieldsGrouping", "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.d.worker - Worker 95ac2248--43a5-8daf-03818ac27a03 for storm StormTopologyFieldsGrouping-- on ed6a51e1-c479-48f2-b386-7d0278d524a4: has finished loading
[Thread-] INFO o.a.s.config - SET worker-user 95ac2248--43a5-8daf-03818ac27a03
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Preparing bolt __acker:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Prepared bolt __acker:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opening spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opened spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Activating spout MySpout:()
spout:
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Preparing bolt __system:(-)
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Prepared bolt __system:(-)
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
Storm的stream grouping的Non Grouping
它是随机分派，意思是说stream不关心到底谁会收到它的tuple。目前它和Shuffle grouping是一样的效果。
编写代码StormTopologyNonGrouping.java
package zhouls.bigdata.stormDemo; import java.util.Map; import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseRichBolt;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils; /**
* NonGrouping
* 字段分组
* @author zhouls
*
*/ public class StormTopologyNonGrouping { public static class MySpout extends BaseRichSpout{
private Map conf;
private TopologyContext context;
private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context,
SpoutOutputCollector collector) {
this.conf = conf;
this.collector = collector;
this.context = context;
} int num = ; public void nextTuple() {
num++;
System.out.println("spout:"+num);
this.collector.emit(new Values(num,num%));
Utils.sleep();
} public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("num","flag"));
} } public static class MyBolt extends BaseRichBolt{ private Map stormConf;
private TopologyContext context;
private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.stormConf = stormConf;
this.context = context;
this.collector = collector;
} public void execute(Tuple input) {
Integer num = input.getIntegerByField("num");
System.err.println("thread:"+Thread.currentThread().getId()+",num="+num);
} public void declareOutputFields(OutputFieldsDeclarer declarer) { } } public static void main(String[] args) {
TopologyBuilder topologyBuilder = new TopologyBuilder();
String spout_id = MySpout.class.getSimpleName();
String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout());
topologyBuilder.setBolt(bolt_id, new MyBolt(),).noneGrouping(spout_id); Config config = new Config();
String topology_name = StormTopologyFieldsGrouping.class.getSimpleName();
if(args.length==){
//在本地运行
LocalCluster localCluster = new LocalCluster();
localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology());
}else{
//在集群运行
try {
StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology());
} catch (AlreadyAliveException e) {
e.printStackTrace();
} catch (InvalidTopologyException e) {
e.printStackTrace();
} catch (AuthorizationException e) {
e.printStackTrace();
}
} } }
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0000 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0000, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0001 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0001, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0002 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0002, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa0002
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa0002 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@40de8f93
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d87351baa0002, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa0002
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@45ab3bdd
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0003, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0003 with negotiated timeout for client /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0004 with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0004, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.zookeeper - Queued up for leader lock.
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d87351baa0001 type:create cxid:0x1 zxid:0x12 txntype:- reqpath:n/a Error Path:/storm/leader-lock Error:KeeperErrorCode = NoNode for /storm/leader-lock
[Curator-Framework-] WARN o.a.s.s.o.a.c.u.ZKPaths - The version of ZooKeeper being used doesn't support Container nodes. CreateMode.PERSISTENT will be used instead.
[main] INFO o.a.s.d.m.MetricsUtils - Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter
[main] INFO o.a.s.d.m.r.JmxPreparableReporter - Preparing...
[main] INFO o.a.s.d.common - Started statistics report plugin...
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@17dad32f
[main-EventThread] INFO o.a.s.zookeeper - WIN-BQOBV63OBNM gained leadership, checking if it has all the topology code locally.
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-EventThread] INFO o.a.s.zookeeper - active-topology-ids [] local-topology-ids [] diff-topology []
[main-EventThread] INFO o.a.s.zookeeper - Accepting leadership, all active topology found localy.
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0005 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0005, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa0005
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa0005 closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa0005
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@7c281eb8
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4a8ffd75
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0006 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0006, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0007 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0007, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa0007
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa0007 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa0007
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@3e05586b
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0008 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0008, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\bd79fa6c-6664-4148-a7fa-8a4bd7eadea3", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\bd79fa6c---a7fa-8a4bd7eadea3\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\bd79fa6c---a7fa-8a4bd7eadea3\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@294aba23
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0009 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0009, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa0009
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa0009 closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa0009
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5f5827d0
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa000a with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa000a, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id f712f019--4de8--c45fe5e6ae08 at host WIN-BQOBV63OBNM
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\55c6dbc6-2288-4cd9-8e09-b65f5aa6048c", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\55c6dbc6--4cd9-8e09-b65f5aa6048c\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\55c6dbc6--4cd9-8e09-b65f5aa6048c\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@c689973
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa000b, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa000b with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa000b
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa000b closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa000b
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@2b148329
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa000c with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa000c, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id a9bca7a4-e0a6-45d9-b3d7-21525100402e at host WIN-BQOBV63OBNM
[main] INFO o.a.s.l.ThriftAccessLogger - Request ID: access from: principal: operation: submitTopology
[main] INFO o.a.s.d.nimbus - Received topology submission for StormTopologyFieldsGrouping with conf {"topology.max.task.parallelism" nil, "topology.submitter.principal" "", "topology.acker.executors" nil, "topology.eventlogger.executors" , "storm.zookeeper.superACL" nil, "topology.users" (), "topology.submitter.user" "Administrator", "topology.kryo.register" nil, "topology.kryo.decorators" (), "storm.id" "StormTopologyFieldsGrouping-1-1501211994", "topology.name" "StormTopologyFieldsGrouping"}
[main] INFO o.a.s.d.nimbus - uploadedJar
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@1d0cac30
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa000d with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa000d, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d87351baa000d type:create cxid:0x2 zxid:0x26 txntype:- reqpath:n/a Error Path:/storm/blobstoremaxkeysequencenumber Error:KeeperErrorCode = NoNode for /storm/blobstoremaxkeysequencenumber
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa000d
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa000d
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa000d closed
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyFieldsGrouping---stormconf.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@1ff15a50
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa000e, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa000e with negotiated timeout for client /127.0.0.1:
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa000e
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa000e closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d87351baa000e, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa000e
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyFieldsGrouping---stormcode.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.d.nimbus - desired replication count achieved, current-replication-count for conf key = , current-replication-count for code key = , current-replication-count for jar key =
[main] INFO o.a.s.d.nimbus - Activating StormTopologyFieldsGrouping: StormTopologyFieldsGrouping--
[timer] INFO o.a.s.s.EvenScheduler - Available slots: (["f712f019-6412-4de8-8018-c45fe5e6ae08" ] ["f712f019-6412-4de8-8018-c45fe5e6ae08" ] ["f712f019-6412-4de8-8018-c45fe5e6ae08" ] ["a9bca7a4-e0a6-45d9-b3d7-21525100402e" ] ["a9bca7a4-e0a6-45d9-b3d7-21525100402e" ] ["a9bca7a4-e0a6-45d9-b3d7-21525100402e" ])
[timer] INFO o.a.s.d.nimbus - Setting new assignment for topology id StormTopologyFieldsGrouping--: #org.apache.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\def58122-150b-46e7-8ba5-516957a542b6", :node->host {"f712f019-6412-4de8-8018-c45fe5e6ae08" "WIN-BQOBV63OBNM"}, :executor->node+port {[ ] ["f712f019-6412-4de8-8018-c45fe5e6ae08" ], [ ] ["f712f019-6412-4de8-8018-c45fe5e6ae08" ], [ ] ["f712f019-6412-4de8-8018-c45fe5e6ae08" ], [ ] ["f712f019-6412-4de8-8018-c45fe5e6ae08" ]}, :executor->start-time-secs {[ ] , [ ] , [ ] , [ ] }, :worker->resources {["f712f019-6412-4de8-8018-c45fe5e6ae08" ] [0.0 0.0 0.0]}}
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\def58122-150b-46e7-8ba5-516957a542b6\blobs
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa000f, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa000f with negotiated timeout for client /127.0.0.1:
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa000f
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d87351baa000f, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa000f
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa000f closed
[Thread-] INFO o.a.s.d.supervisor - Removing code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyFieldsGrouping-1-1501211994", :executors [[4 4] [3 3] [2 2] [1 1]], :resources #object[org.apache.storm.generated.WorkerResources 0x316130d1 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor f712f019-6412-4de8-8018-c45fe5e6ae08 on port 1024 with id 0ceb31d3-5aa2-4ac3-aa14-bb416fcf2b5b
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyFieldsGrouping-1-1501211994", :executors [[4 4] [3 3] [2 2] [1 1]], :resources #object[org.apache.storm.generated.WorkerResources 0x5647bf70 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor f712f019-6412-4de8-8018-c45fe5e6ae08 on port 1024 with id aa3149ee-84d2-4c7b-ad40-25331c8b5c4f
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyFieldsGrouping-1-1501211994", :executors [[4 4] [3 3] [2 2] [1 1]], :resources #object[org.apache.storm.generated.WorkerResources 0x6554a4c0 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor f712f019-6412-4de8-8018-c45fe5e6ae08 on port 1024 with id b093df1f-6182-478e-b274-7c51c73c3e9c
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyFieldsGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@4acb69bd
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\def58122-150b-46e7-8ba5-516957a542b6\blobs
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0010 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0010, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa0010
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa0010
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa0010 closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.d.supervisor - Launching worker with assignment {:storm-id "StormTopologyFieldsGrouping-1-1501211994", :executors [[ ] [ ] [ ] [ ]], :resources #object[org.apache.storm.generated.WorkerResources 0x5d8bf703 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor f712f019--4de8--c45fe5e6ae08 on port with id e0fd30c7-80aa-4c5b-85b9-36af5e0b7089
[Thread-] INFO o.a.s.d.worker - Launching worker for StormTopologyFieldsGrouping-- on f712f019--4de8--c45fe5e6ae08: with id e0fd30c7-80aa-4c5b-85b9-36af5e0b7089 and conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\bd79fa6c-6664-4148-a7fa-8a4bd7eadea3", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5fd07a9
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyFieldsGrouping--
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0011 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0011, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread--EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d87351baa0011
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d87351baa0011 closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d87351baa0011
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@65d23be8
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d87351baa0012 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d87351baa0012, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread-] INFO o.a.s.s.a.AuthUtils - Got AutoCreds []
[Thread-] INFO o.a.s.d.worker - Reading Assignments.
[Thread-] INFO o.a.s.d.worker - Registering IConnectionCallbacks for f712f019--4de8--c45fe5e6ae08:
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[refresh-active-timer] INFO o.a.s.d.worker - All connections are ready for worker f712f019--4de8--c45fe5e6ae08: with id e0fd30c7-80aa-4c5b-85b9-36af5e0b7089
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __system:[- -]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Timeouts disabled for executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.worker - Started with log levels: {"" #object[org.apache.logging.log4j.Level 0x575c9922 "INFO"], "org.apache.zookeeper" #object[org.apache.logging.log4j.Level 0x41aeb83c "WARN"]}
[Thread-] INFO o.a.s.d.worker - Worker has topology config {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "topology.submitter.principal" "", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\def58122-150b-46e7-8ba5-516957a542b6", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "storm.zookeeper.superACL" nil, "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" [ ], "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "topology.users" [], "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.submitter.user" "Administrator", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "topology.kryo.register" nil, "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "topology.kryo.decorators" [], "storm.id" "StormTopologyFieldsGrouping-1-1501211994", "topology.name" "StormTopologyFieldsGrouping", "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.d.worker - Worker e0fd30c7-80aa-4c5b-85b9-36af5e0b7089 for storm StormTopologyFieldsGrouping-- on f712f019--4de8--c45fe5e6ae08: has finished loading
[Thread-] INFO o.a.s.config - SET worker-user e0fd30c7-80aa-4c5b-85b9-36af5e0b7089
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opening spout MySpout:()
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Preparing bolt __system:(-)
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opened spout MySpout:()
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Prepared bolt __system:(-)
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Activating spout MySpout:()
spout:
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Preparing bolt __acker:()
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Prepared bolt __acker:()
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1424ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1359ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1916ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1859ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1971ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2380ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1409ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2299ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2410ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1824ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2781ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2054ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1407ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2766ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1643ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num= [SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2413ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1735ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1854ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
thread:,num=
spout:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1498ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2533ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2815ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
Storm的stream grouping的Direct Grouping
它是直接分组，这是一种比较特别的分组方法，用这种分组一味着消息的发送者具体由消息接受者的哪个tasl处理这个消息。只有被声明为Direct Steam的消息流可以声明这种分组方法。而且这种消息tuple必须使用emitDirect方法来发射。消息处理者可以通过TopologyContext来或者处理它的消息的taskid（OutputCollector.emit方法也会返回taskid）
编写代码StormTopologyDirectGrouping.java
package zhouls.bigdata.stormDemo; import java.util.Map; import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseRichBolt;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils; /**
* DirectGrouping
* 直接分组
* @author zhouls
*
*/ public class StormTopologyDirectGrouping { public static class MySpout extends BaseRichSpout{
private Map conf;
private TopologyContext context;
private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context,
SpoutOutputCollector collector) {
this.conf = conf;
this.collector = collector;
this.context = context;
} int num = ; public void nextTuple() {
num++;
System.out.println("spout:"+num);
this.collector.emit(new Values(num,num%));
Utils.sleep();
} public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("num","flag"));
} } public static class MyBolt extends BaseRichBolt{ private Map stormConf;
private TopologyContext context;
private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.stormConf = stormConf;
this.context = context;
this.collector = collector;
} public void execute(Tuple input) {
Integer num = input.getIntegerByField("num");
System.err.println("thread:"+Thread.currentThread().getId()+",num="+num);
} public void declareOutputFields(OutputFieldsDeclarer declarer) { } } public static void main(String[] args) {
TopologyBuilder topologyBuilder = new TopologyBuilder();
String spout_id = MySpout.class.getSimpleName();
String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout());
topologyBuilder.setBolt(bolt_id, new MyBolt(),).directGrouping(spout_id); Config config = new Config();
String topology_name = StormTopologyFieldsGrouping.class.getSimpleName();
if(args.length==){
//在本地运行
LocalCluster localCluster = new LocalCluster();
localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology());
}else{
//在集群运行
try {
StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology());
} catch (AlreadyAliveException e) {
e.printStackTrace();
} catch (InvalidTopologyException e) {
e.printStackTrace();
} catch (AuthorizationException e) {
e.printStackTrace();
}
} } }
Storm的stream grouping的LocalOrShuffer Grouping
编写代码
StormTopologyLocalOrShufferGrouping.java
package zhouls.bigdata.stormDemo; import java.util.Map; import org.apache.storm.Config;
import org.apache.storm.LocalCluster;
import org.apache.storm.StormSubmitter;
import org.apache.storm.generated.AlreadyAliveException;
import org.apache.storm.generated.AuthorizationException;
import org.apache.storm.generated.InvalidTopologyException;
import org.apache.storm.spout.SpoutOutputCollector;
import org.apache.storm.task.OutputCollector;
import org.apache.storm.task.TopologyContext;
import org.apache.storm.topology.OutputFieldsDeclarer;
import org.apache.storm.topology.TopologyBuilder;
import org.apache.storm.topology.base.BaseRichBolt;
import org.apache.storm.topology.base.BaseRichSpout;
import org.apache.storm.tuple.Fields;
import org.apache.storm.tuple.Tuple;
import org.apache.storm.tuple.Values;
import org.apache.storm.utils.Utils; /**
* LocalAllshufferGrouping
* 本地分组或随机分组
* @author zhouls
*
*/ public class StormTopologyLocalOrShufferGrouping { public static class MySpout extends BaseRichSpout{
private Map conf;
private TopologyContext context;
private SpoutOutputCollector collector; public void open(Map conf, TopologyContext context,
SpoutOutputCollector collector) {
this.conf = conf;
this.collector = collector;
this.context = context;
} int num = ; public void nextTuple() {
num++;
System.out.println("spout:"+num);
this.collector.emit(new Values(num));
Utils.sleep();
} public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("num"));
} } public static class MyBolt extends BaseRichBolt{ private Map stormConf;
private TopologyContext context;
private OutputCollector collector; public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.stormConf = stormConf;
this.context = context;
this.collector = collector;
} public void execute(Tuple input) {
Integer num = input.getIntegerByField("num");
System.err.println("thread:"+Thread.currentThread().getId()+",num="+num);
} public void declareOutputFields(OutputFieldsDeclarer declarer) { } } public static void main(String[] args) {
TopologyBuilder topologyBuilder = new TopologyBuilder();
String spout_id = MySpout.class.getSimpleName();
String bolt_id = MyBolt.class.getSimpleName(); topologyBuilder.setSpout(spout_id, new MySpout());
topologyBuilder.setBolt(bolt_id, new MyBolt(),).localOrShuffleGrouping(spout_id); Config config = new Config();
config.setNumWorkers();
String topology_name = StormTopologyLocalOrShufferGrouping.class.getSimpleName();
if(args.length==){
//在本地运行
LocalCluster localCluster = new LocalCluster();
localCluster.submitTopology(topology_name, config, topologyBuilder.createTopology());
}else{
//在集群运行
try {
StormSubmitter.submitTopology(topology_name, config, topologyBuilder.createTopology());
} catch (AlreadyAliveException e) {
e.printStackTrace();
} catch (InvalidTopologyException e) {
e.printStackTrace();
} catch (AuthorizationException e) {
e.printStackTrace();
}
} } }
停掉，我们粘贴来分析分析
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\c90c00fb-b500-47a1-a096-274b0b11e570", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\c90c00fb-b500-47a1-a096-274b0b11e570\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\c90c00fb-b500-47a1-a096-274b0b11e570\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@69909c14
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2142ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a40009 with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a40009, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a40009
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2197ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a40009 closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a40009
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@58a2d9f9
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1578ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a4000a with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a4000a, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1715ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[main] INFO o.a.s.d.supervisor - Starting supervisor with id ad5c5ae8-109f-40ce-822c-d8890af7e09d at host WIN-BQOBV63OBNM
[main] INFO o.a.s.d.supervisor - Starting Supervisor with conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\1daf2b63-c9b0-4081-b270-e93a0ee3da4b", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[main] INFO o.a.s.l.Localizer - Reconstruct localized resource: C:\Users\ADMINI~\AppData\Local\Temp\1daf2b63-c9b0--b270-e93a0ee3da4b\supervisor\usercache
[main] WARN o.a.s.l.Localizer - No left over resources found for any user during reconstructing of local resources at: C:\Users\ADMINI~\AppData\Local\Temp\1daf2b63-c9b0--b270-e93a0ee3da4b\supervisor\usercache
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@c689973
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a4000b with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a4000b, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main-EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a4000b
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a4000b
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a4000b closed
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@2b148329
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a4000c with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a4000c, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[main] INFO o.a.s.d.supervisor - Starting supervisor with id 27640a2c-81b0--a9fb-0d4678bec8d6 at host WIN-BQOBV63OBNM
[main] INFO o.a.s.l.ThriftAccessLogger - Request ID: access from: principal: operation: submitTopology
[main] INFO o.a.s.d.nimbus - Received topology submission for StormTopologyLocalOrShufferGrouping with conf {"topology.max.task.parallelism" nil, "topology.submitter.principal" "", "topology.acker.executors" nil, "topology.eventlogger.executors" , "topology.workers" , "storm.zookeeper.superACL" nil, "topology.users" (), "topology.submitter.user" "Administrator", "topology.kryo.register" nil, "topology.kryo.decorators" (), "storm.id" "StormTopologyLocalOrShufferGrouping-1-1501214563", "topology.name" "StormTopologyLocalOrShufferGrouping"}
[main] INFO o.a.s.d.nimbus - uploadedJar
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5b39a3e6
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a4000d with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a4000d, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x15d875b69a4000d type:create cxid:0x2 zxid:0x27 txntype:- reqpath:n/a Error Path:/storm/blobstoremaxkeysequencenumber Error:KeeperErrorCode = NoNode for /storm/blobstoremaxkeysequencenumber
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a4000d
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a4000d
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a4000d closed
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyLocalOrShufferGrouping---stormconf.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@164642a4
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a4000e with negotiated timeout for client /127.0.0.1:
[main-SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a4000e, negotiated timeout =
[main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a4000e
[main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a4000e closed
[main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a4000e
[main] INFO o.a.s.cluster - setup-path/blobstore/StormTopologyLocalOrShufferGrouping---stormcode.ser/WIN-BQOBV63OBNM:-
[main] INFO o.a.s.d.nimbus - desired replication count achieved, current-replication-count for conf key = , current-replication-count for code key = , current-replication-count for jar key =
[main] INFO o.a.s.d.nimbus - Activating StormTopologyLocalOrShufferGrouping: StormTopologyLocalOrShufferGrouping--
[timer] INFO o.a.s.s.EvenScheduler - Available slots: (["ad5c5ae8-109f-40ce-822c-d8890af7e09d" ] ["ad5c5ae8-109f-40ce-822c-d8890af7e09d" ] ["ad5c5ae8-109f-40ce-822c-d8890af7e09d" ] ["27640a2c-81b0-4093-a9fb-0d4678bec8d6" ] ["27640a2c-81b0-4093-a9fb-0d4678bec8d6" ] ["27640a2c-81b0-4093-a9fb-0d4678bec8d6" ])
[timer] INFO o.a.s.d.nimbus - Setting new assignment for topology id StormTopologyLocalOrShufferGrouping--: #org.apache.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\848a3a03-1541-45bd-a398-e7b243d1ae97", :node->host {"27640a2c-81b0-4093-a9fb-0d4678bec8d6" "WIN-BQOBV63OBNM", "ad5c5ae8-109f-40ce-822c-d8890af7e09d" "WIN-BQOBV63OBNM"}, :executor->node+port {[ ] ["27640a2c-81b0-4093-a9fb-0d4678bec8d6" ], [ ] ["ad5c5ae8-109f-40ce-822c-d8890af7e09d" ], [ ] ["27640a2c-81b0-4093-a9fb-0d4678bec8d6" ], [ ] ["27640a2c-81b0-4093-a9fb-0d4678bec8d6" ], [ ] ["ad5c5ae8-109f-40ce-822c-d8890af7e09d" ], [ ] ["ad5c5ae8-109f-40ce-822c-d8890af7e09d" ]}, :executor->start-time-secs {[ ] , [ ] , [ ] , [ ] , [ ] , [ ] }, :worker->resources {["27640a2c-81b0-4093-a9fb-0d4678bec8d6" ] [0.0 0.0 0.0], ["ad5c5ae8-109f-40ce-822c-d8890af7e09d" ] [0.0 0.0 0.0]}}
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyLocalOrShufferGrouping--
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyLocalOrShufferGrouping--
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@9b36612
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\848a3a03--45bd-a398-e7b243d1ae97\blobs
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\848a3a03--45bd-a398-e7b243d1ae97\blobs
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a4000f with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a4000f, negotiated timeout =
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a4000f
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a40010 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a40010, negotiated timeout =
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a40010
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a4000f
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a4000f closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a40010 closed
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x15d875b69a40010, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:) [storm-core-1.0..jar:1.0.]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:) [storm-core-1.0..jar:1.0.]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_66]
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a40010
[Thread-] INFO o.a.s.d.supervisor - Removing code for storm id StormTopologyLocalOrShufferGrouping--
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyLocalOrShufferGrouping--
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyLocalOrShufferGrouping--
[Thread-] INFO o.a.s.d.supervisor - Launching worker with assignment {:storm-id "StormTopologyLocalOrShufferGrouping-1-1501214563", :executors [[ ] [ ] [ ]], :resources #object[org.apache.storm.generated.WorkerResources 0x394b4bb1 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor 27640a2c-81b0--a9fb-0d4678bec8d6 on port with id ffa50ee3-fbb4--bd13-63799ccd8b93
[Thread-] INFO o.a.s.d.worker - Launching worker for StormTopologyLocalOrShufferGrouping-- on 27640a2c-81b0--a9fb-0d4678bec8d6: with id ffa50ee3-fbb4--bd13-63799ccd8b93 and conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\1daf2b63-c9b0-4081-b270-e93a0ee3da4b", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@5964b4fe
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyLocalOrShufferGrouping-1-1501214563", :executors [[3 3] [1 1] [5 5]], :resources #object[org.apache.storm.generated.WorkerResources 0x3646ae7b "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ad5c5ae8-109f-40ce-822c-d8890af7e09d on port 1024 with id f23a724c-99c9-4ee1-9d0b-e054bd11e132
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a40011 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a40011, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread--EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a40011
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a40011
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a40011 closed
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@33d8afd
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a40012 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a40012, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread-] INFO o.a.s.s.a.AuthUtils - Got AutoCreds []
[Thread-] INFO o.a.s.d.worker - Reading Assignments.
[Thread-] INFO o.a.s.d.worker - Registering IConnectionCallbacks for 27640a2c-81b0--a9fb-0d4678bec8d6:
[Thread-] INFO o.a.s.d.executor - Loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MyBolt:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Timeouts disabled for executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __acker:[ ]
[Thread-] INFO o.a.s.d.executor - Loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks __system:[- -]
[Thread-] INFO o.a.s.d.executor - Finished loading executor __system:[- -]
[Thread-] INFO o.a.s.d.executor - Loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Loaded executor tasks MySpout:[ ]
[Thread-] INFO o.a.s.d.executor - Finished loading executor MySpout:[ ]
[Thread-] INFO o.a.s.d.worker - Started with log levels: {"" #object[org.apache.logging.log4j.Level 0x21000bbd "INFO"], "org.apache.zookeeper" #object[org.apache.logging.log4j.Level 0x6150d5e3 "WARN"]}
[Thread-] INFO o.a.s.d.worker - Worker has topology config {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "topology.submitter.principal" "", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\848a3a03-1541-45bd-a398-e7b243d1ae97", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "storm.zookeeper.superACL" nil, "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" [ ], "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "topology.users" [], "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.submitter.user" "Administrator", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "topology.kryo.register" nil, "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "topology.kryo.decorators" [], "storm.id" "StormTopologyLocalOrShufferGrouping-1-1501214563", "topology.name" "StormTopologyLocalOrShufferGrouping", "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.d.worker - Worker ffa50ee3-fbb4--bd13-63799ccd8b93 for storm StormTopologyLocalOrShufferGrouping-- on 27640a2c-81b0--a9fb-0d4678bec8d6: has finished loading
[Thread-] INFO o.a.s.config - SET worker-user ffa50ee3-fbb4--bd13-63799ccd8b93
[refresh-active-timer] INFO o.a.s.d.worker - All connections are ready for worker 27640a2c-81b0--a9fb-0d4678bec8d6: with id ffa50ee3-fbb4--bd13-63799ccd8b93
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Preparing bolt __acker:()
[Thread--__acker-executor[ ]] INFO o.a.s.d.executor - Prepared bolt __acker:()
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Preparing bolt __system:(-)
[Thread--__system-executor[- -]] INFO o.a.s.d.executor - Prepared bolt __system:(-)
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Preparing bolt MyBolt:()
[Thread--MyBolt-executor[ ]] INFO o.a.s.d.executor - Prepared bolt MyBolt:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opening spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Opened spout MySpout:()
[Thread--MySpout-executor[ ]] INFO o.a.s.d.executor - Activating spout MySpout:()
spout:
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyLocalOrShufferGrouping-1-1501214563", :executors [[3 3] [1 1] [5 5]], :resources #object[org.apache.storm.generated.WorkerResources 0x1146b72f "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ad5c5ae8-109f-40ce-822c-d8890af7e09d on port 1024 with id 42e1e5f8-78e5-402b-83cd-7f87c72fc86f
spout:
thread:,num=
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3138ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.d.supervisor - Downloading code for storm id StormTopologyLocalOrShufferGrouping--
spout:
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@44f30dfd
[Thread-] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in C:\Users\ADMINI~\AppData\Local\Temp\848a3a03--45bd-a398-e7b243d1ae97\blobs
thread:,num=
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyLocalOrShufferGrouping-1-1501214563", :executors [[3 3] [1 1] [5 5]], :resources #object[org.apache.storm.generated.WorkerResources 0x456a5f34 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ad5c5ae8-109f-40ce-822c-d8890af7e09d on port 1024 with id d185964d-4ecc-46e6-8abf-df508bb1b3bd
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 3409ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyLocalOrShufferGrouping-1-1501214563", :executors [[3 3] [1 1] [5 5]], :resources #object[org.apache.storm.generated.WorkerResources 0x46dc1494 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ad5c5ae8-109f-40ce-822c-d8890af7e09d on port 1024 with id 544e4331-9f0f-4b57-89b3-8dc92c43ab75
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1760ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a40013, negotiated timeout =
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a40013 with negotiated timeout for client /127.0.0.1:
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a40013
spout:
thread:,num=
[Thread-] INFO o.a.s.d.supervisor - Missing topology storm code, so can't launch worker with assignment {:storm-id "StormTopologyLocalOrShufferGrouping-1-1501214563", :executors [[3 3] [1 1] [5 5]], :resources #object[org.apache.storm.generated.WorkerResources 0x1edf0f93 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ad5c5ae8-109f-40ce-822c-d8890af7e09d on port 1024 with id f6b866be-aabf-437f-bafa-0fb926fe975f
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1981ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a40013 closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a40013
spout:
thread:,num=
spout:
thread:,num=
[Thread-] INFO o.a.s.d.supervisor - Launching worker with assignment {:storm-id "StormTopologyLocalOrShufferGrouping-1-1501214563", :executors [[ ] [ ] [ ]], :resources #object[org.apache.storm.generated.WorkerResources 0x3a842ca8 "WorkerResources(mem_on_heap:0.0, mem_off_heap:0.0, cpu:0.0)"]} for this supervisor ad5c5ae8-109f-40ce-822c-d8890af7e09d on port with id ada843e9--42bb-beb4-00d1c4735e38
[Thread-] INFO o.a.s.d.worker - Launching worker for StormTopologyLocalOrShufferGrouping-- on ad5c5ae8-109f-40ce-822c-d8890af7e09d: with id ada843e9--42bb-beb4-00d1c4735e38 and conf {"topology.builtin.metrics.bucket.size.secs" , "nimbus.childopts" "-Xmx1024m", "ui.filter.params" nil, "storm.cluster.mode" "local", "storm.messaging.netty.client_worker_threads" , "logviewer.max.per.worker.logs.size.mb" , "supervisor.run.worker.as.user" false, "topology.max.task.parallelism" nil, "topology.priority" , "zmq.threads" , "storm.group.mapping.service" "org.apache.storm.security.auth.ShellBasedGroupsMapping", "transactional.zookeeper.root" "/transactional", "topology.sleep.spout.wait.strategy.time.ms" , "scheduler.display.resource" false, "topology.max.replication.wait.time.sec" , "drpc.invocations.port" , "supervisor.localizer.cache.target.size.mb" , "topology.multilang.serializer" "org.apache.storm.multilang.JsonSerializer", "storm.messaging.netty.server_worker_threads" , "nimbus.blobstore.class" "org.apache.storm.blobstore.LocalFsBlobStore", "resource.aware.scheduler.eviction.strategy" "org.apache.storm.scheduler.resource.strategies.eviction.DefaultEvictionStrategy", "topology.max.error.report.per.interval" , "storm.thrift.transport" "org.apache.storm.security.auth.SimpleTransportPlugin", "zmq.hwm" , "storm.group.mapping.service.params" nil, "worker.profiler.enabled" false, "storm.principal.tolocal" "org.apache.storm.security.auth.DefaultPrincipalToLocal", "supervisor.worker.shutdown.sleep.secs" , "pacemaker.host" "localhost", "storm.zookeeper.retry.times" , "ui.actions.enabled" true, "zmq.linger.millis" , "supervisor.enable" true, "topology.stats.sample.rate" 0.05, "storm.messaging.netty.min_wait_ms" , "worker.log.level.reset.poll.secs" , "storm.zookeeper.port" , "supervisor.heartbeat.frequency.secs" , "topology.enable.message.timeouts" true, "supervisor.cpu.capacity" 400.0, "drpc.worker.threads" , "supervisor.blobstore.download.thread.count" , "drpc.queue.size" , "topology.backpressure.enable" false, "supervisor.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "storm.blobstore.inputstream.buffer.size.bytes" , "topology.shellbolt.max.pending" , "drpc.https.keystore.password" "", "nimbus.code.sync.freq.secs" , "logviewer.port" , "topology.scheduler.strategy" "org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy", "topology.executor.send.buffer.size" , "resource.aware.scheduler.priority.strategy" "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy", "pacemaker.auth.method" "NONE", "storm.daemon.metrics.reporter.plugins" ["org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter"], "topology.worker.logwriter.childopts" "-Xmx64m", "topology.spout.wait.strategy" "org.apache.storm.spout.SleepSpoutWaitStrategy", "ui.host" "0.0.0.0", "storm.nimbus.retry.interval.millis" , "nimbus.inbox.jar.expiration.secs" , "dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.acker.executors" nil, "topology.fall.back.on.java.serialization" true, "topology.eventlogger.executors" , "supervisor.localizer.cleanup.interval.ms" , "storm.zookeeper.servers" ["localhost"], "nimbus.thrift.threads" , "logviewer.cleanup.age.mins" , "topology.worker.childopts" nil, "topology.classpath" nil, "supervisor.monitor.frequency.secs" , "nimbus.credential.renewers.freq.secs" , "topology.skip.missing.kryo.registrations" true, "drpc.authorizer.acl.filename" "drpc-auth-acl.yaml", "pacemaker.kerberos.users" [], "storm.group.mapping.service.cache.duration.secs" , "topology.testing.always.try.serialize" false, "nimbus.monitor.freq.secs" , "storm.health.check.timeout.ms" , "supervisor.supervisors" [], "topology.tasks" nil, "topology.bolts.outgoing.overflow.buffer.enable" false, "storm.messaging.netty.socket.backlog" , "topology.workers" , "pacemaker.base.threads" , "storm.local.dir" "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\c90c00fb-b500-47a1-a096-274b0b11e570", "topology.disable.loadaware" false, "worker.childopts" "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=1M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump", "storm.auth.simple-white-list.users" [], "topology.disruptor.batch.timeout.millis" , "topology.message.timeout.secs" , "topology.state.synchronization.timeout.secs" , "topology.tuple.serializer" "org.apache.storm.serialization.types.ListDelegateSerializer", "supervisor.supervisors.commands" [], "nimbus.blobstore.expiration.secs" , "logviewer.childopts" "-Xmx128m", "topology.environment" nil, "topology.debug" false, "topology.disruptor.batch.size" , "storm.messaging.netty.max_retries" , "ui.childopts" "-Xmx768m", "storm.network.topography.plugin" "org.apache.storm.networktopography.DefaultRackDNSToSwitchMapping", "storm.zookeeper.session.timeout" , "drpc.childopts" "-Xmx768m", "drpc.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.connection.timeout" , "storm.zookeeper.auth.user" nil, "storm.meta.serialization.delegate" "org.apache.storm.serialization.GzipThriftSerializationDelegate", "topology.max.spout.pending" nil, "storm.codedistributor.class" "org.apache.storm.codedistributor.LocalFileSystemCodeDistributor", "nimbus.supervisor.timeout.secs" , "nimbus.task.timeout.secs" , "drpc.port" , "pacemaker.max.threads" , "storm.zookeeper.retry.intervalceiling.millis" , "nimbus.thrift.port" , "storm.auth.simple-acl.admins" [], "topology.component.cpu.pcore.percent" 10.0, "supervisor.memory.capacity.mb" 3072.0, "storm.nimbus.retry.times" , "supervisor.worker.start.timeout.secs" , "storm.zookeeper.retry.interval" , "logs.users" nil, "worker.profiler.command" "flight.bash", "transactional.zookeeper.port" nil, "drpc.max_buffer_size" , "pacemaker.thread.timeout" , "task.credentials.poll.secs" , "blobstore.superuser" "Administrator", "drpc.https.keystore.type" "JKS", "topology.worker.receiver.thread.count" , "topology.state.checkpoint.interval.ms" , "supervisor.slots.ports" ( ), "topology.transfer.buffer.size" , "storm.health.check.dir" "healthchecks", "topology.worker.shared.thread.pool.size" , "drpc.authorizer.acl.strict" false, "nimbus.file.copy.expiration.secs" , "worker.profiler.childopts" "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder", "topology.executor.receive.buffer.size" , "backpressure.disruptor.low.watermark" 0.4, "nimbus.task.launch.secs" , "storm.local.mode.zmq" false, "storm.messaging.netty.buffer_size" , "storm.cluster.state.store" "org.apache.storm.cluster_state.zookeeper_state_factory", "worker.heartbeat.frequency.secs" , "storm.log4j2.conf.dir" "log4j2", "ui.http.creds.plugin" "org.apache.storm.security.auth.DefaultHttpCredentialsPlugin", "storm.zookeeper.root" "/storm", "topology.tick.tuple.freq.secs" nil, "drpc.https.port" -, "storm.workers.artifacts.dir" "workers-artifacts", "supervisor.blobstore.download.max_retries" , "task.refresh.poll.secs" , "storm.exhibitor.port" , "task.heartbeat.frequency.secs" , "pacemaker.port" , "storm.messaging.netty.max_wait_ms" , "topology.component.resources.offheap.memory.mb" 0.0, "drpc.http.port" , "topology.error.throttle.interval.secs" , "storm.messaging.transport" "org.apache.storm.messaging.netty.Context", "storm.messaging.netty.authentication" false, "topology.component.resources.onheap.memory.mb" 128.0, "topology.kryo.factory" "org.apache.storm.serialization.DefaultKryoFactory", "worker.gc.childopts" "", "nimbus.topology.validator" "org.apache.storm.nimbus.DefaultTopologyValidator", "nimbus.seeds" ["localhost"], "nimbus.queue.size" , "nimbus.cleanup.inbox.freq.secs" , "storm.blobstore.replication.factor" , "worker.heap.memory.mb" , "logviewer.max.sum.worker.logs.size.mb" , "pacemaker.childopts" "-Xmx1024m", "ui.users" nil, "transactional.zookeeper.servers" nil, "supervisor.worker.timeout.secs" , "storm.zookeeper.auth.password" nil, "storm.blobstore.acl.validation.enabled" false, "client.blobstore.class" "org.apache.storm.blobstore.NimbusBlobStore", "supervisor.childopts" "-Xmx256m", "topology.worker.max.heap.size.mb" 768.0, "ui.http.x-frame-options" "DENY", "backpressure.disruptor.high.watermark" 0.9, "ui.filter" nil, "ui.header.buffer.bytes" , "topology.min.replication.count" , "topology.disruptor.wait.timeout.millis" , "storm.nimbus.retry.intervalceiling.millis" , "topology.trident.batch.emit.interval.millis" , "storm.auth.simple-acl.users" [], "drpc.invocations.threads" , "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "ui.port" , "storm.exhibitor.poll.uripath" "/exhibitor/v1/cluster/list", "storm.messaging.netty.transfer.batch.size" , "logviewer.appender.name" "A1", "nimbus.thrift.max_buffer_size" , "storm.auth.simple-acl.users.commands" [], "drpc.request.timeout.secs" }
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost: sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@76613bab
[Thread-] INFO o.a.s.d.supervisor - Finished downloading code for storm id StormTopologyLocalOrShufferGrouping--
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2401ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 1312ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[SyncThread:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x15d875b69a40014 with negotiated timeout for client /127.0.0.1:
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:, sessionid = 0x15d875b69a40014, negotiated timeout =
[Thread--EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
[Thread--EventThread] INFO o.a.s.zookeeper - Zookeeper state update: :connected:none
thread:,num=
spout:
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2294ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2914ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
[Curator-Framework-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
thread:,num=
[ProcessThread(sid: cport:-):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: 0x15d875b69a40014
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2257ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1: which had sessionid 0x15d875b69a40014
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x15d875b69a40014 closed
[Thread--EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
[Thread-] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
[Thread-] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:/storm sessionTimeout= watcher=org.apache.storm.shade.org.apache.curator.ConnectionState@e995f63
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:. Will not attempt to authenticate using SASL (unknown error)
[Thread--SendThread(127.0.0.1:)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:, initiating session
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:
spout:
thread:,num=
spout:
thread:,num=
[SyncThread:] WARN o.a.s.s.o.a.z.s.p.FileTxnLog - fsync-ing the write ahead log in SyncThread: took 2114ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
spout:
thread:,num=
spout:
thread:,num=
spout:
thread:,num=
可以看到只有thread:126接收数据。
即
topologyBuilder.setSpout(spout_id, new MySpout());
topologyBuilder.setBolt(bolt_id, new MyBolt(),).localOrShuffleGrouping(spout_id); Config config = new Config();
config.setNumWorkers();
Storm的stream grouping的CustomStream Grouping
这个自定义storm的流分组，后续更新
Storm编程入门API系列之Storm的Topology的stream grouping的更多相关文章
- Storm编程入门API系列之Storm的Topology多个Workers数目控制实现
前期博客 Storm编程入门API系列之Storm的Topology默认Workers.默认executors和默认tasks数目 继续编写 StormTopologyMoreWorker.java ...
- Storm编程入门API系列之Storm的Topology多个Executors数目控制实现
前期博客 Storm编程入门API系列之Storm的Topology默认Workers.默认executors和默认tasks数目 Storm编程入门API系列之Storm的Topology多个Wor ...
- Storm编程入门API系列之Storm的Topology多个tasks数目控制实现
前期博客 Storm编程入门API系列之Storm的Topology默认Workers.默认executors和默认tasks数目 Storm编程入门API系列之Storm的Topology多个Wor ...
- Storm编程入门API系列之Storm的定时任务实现
概念,见博客 Storm概念学习系列之storm的定时任务 Storm的定时任务,分为两种实现方式,都是可以达到目的的. 我这里,分为StormTopologyTimer1.java 和 Sto ...
- Storm编程入门API系列之Storm的可靠性的ACK消息确认机制
概念,见博客 Storm概念学习系列之storm的可靠性 什么业务场景需要storm可靠性的ACK确认机制? 答:想要保住数据不丢,或者保住数据总是被处理.即若没被处理的,得让我们知道. publi ...
- Storm编程入门API系列之Storm的Topology默认Workers、默认executors和默认tasks数目
关于,storm的启动我这里不多说了. 见博客 storm的3节点集群详细启动步骤(非HA和HA)(图文详解) 建立stormDemo项目 Group Id : zhouls.bigdata Art ...
- Storm概念学习系列之Stream消息流 和 Stream Grouping 消息流组
不多说,直接上干货! Stream消息流是Storm中最关键的抽象,是一个没有边界的Tuple序列. Stream Grouping 消息流组是用来定义一个流如何分配到Tuple到Bolt. Stre ...
- Storm概念学习系列之storm的定时任务
不多说,直接上干货! 至于为什么,有storm的定时任务.这个很简单.但是,这个在工作中非常重要! 假设有如下的业务场景 这个spoult源源不断地发送数据,boilt呢会进行处理.然后呢,处理后的结 ...
- Storm概念学习系列之storm的可靠性
这个概念,对于理解storm很有必要. 1.worker进程死掉 worker是真实存在的.可以jps查看. 正是因为有了storm的可靠性,所以storm会重新启动一个新的worker进程. 2.s ...
随机推荐
- POI2014
...一个shabi和一堆神题的故事 今天只写了两道 之后随缘更吧 啊 顺便 snake我是不会更的 bzoj3829 POI2014 Farmcraft mhy住在一棵有n个点的树的1号结点上,每个 ...
- POJ3352(连通分量缩点)
Road Construction Time Limit: 2000MS Memory Limit: 65536K Total Submissions: 10352 Accepted: 514 ...
- AAAAAA
有可能被立案调查.暂停上市.退市风险警示*ST.特别处理ST的公司:银鸽投资(SH:600069).天山生物(SZ:300313).金贵银业(SZ:002716).美盛文化(SZ:002699).未名 ...
- plsql 分页
select * from (select rownum rn,ps.* from (select * from user_t) ps ) where rn>=1 and rn<=10 ...
- [转]C/C++获取当前系统时间
原文转自:http://www.cnblogs.com/mfryf/archive/2012/02/13/2349360.html 个人觉得第二种还是比较实用的,而且也是最常用的~ 不过当计算算法耗时 ...
- shader之texture
纹理坐标作为属性传递到顶点着色器 texture是OPENGL对象,包含一张或多张相同格式的图片. 它有2中用途: the source of a texture access from a Shad ...
- Hash和Salt Umbraco 默认的password存储方式
本文章转载自 http://blog.reneorban.com/2014/10/hash-and-salt-umbraco-passwords.html Hash and Salt Umbraco ...
- Swoole 协程与 Go 协程的区别
Swoole 协程与 Go 协程的区别 进程.线程.协程的概念 进程是什么? 进程就是应用程序的启动实例. 例如:打开一个软件,就是开启了一个进程. 进程拥有代码和打开的文件资源,数据资源,独立的内存 ...
- 【问题总结】万万没想到，竟然栽在了List手里
说明 昨天同事开发的时候遇到了一个奇怪的问题. 使用Guava做缓存,往里面存一个List,为了方便描述,称它为列表A,在另一个地方取出来,再跟列表B中的元素进行差集处理,简单来说,就像是下面这样: ...
- OVS编译
下载源码 # git clone https://github.com/openvswitch/ovs.git # cd ovs # git checkout branch-2.8 下载依赖包 # y ...