前言

由于项目需要涉及到zookeeper和Kafka的使用，快速做了一篇笔记，方便小伙伴们搭建环境。

zookeeper

官方定义

What is ZooKeeper?

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them, which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

ZooKeeper是一个分布式的，开放源码的分布式应用程序协调服务，是Google的Chubby一个开源的实现，是Hadoop和Hbase的重要组件。它是一个为分布式应用提供一致性服务的软件，提供的功能包括：配置维护、域名服务、分布式同步、组服务等。

ZooKeeper的目标就是封装好复杂易出错的关键服务，将简单易用的接口和性能高效、功能稳定的系统提供给用户。

zookeeper常用端口号：2181

环境搭建

  1. 下载zookeeper：

    下载地址：http://mirrors.hust.edu.cn/apache/zookeeper/

  2. 在zookeeper-3.4.10目录下，新建文件夹，并命名，例如data。(路径为：D:\zookeeper-3.4.10\conf\data)

  3. 复制“zoo_sample.cfg”副本à并将副本重命名为“zoo.cfg，在任意文本编辑器（eg：记事本）中打开zoo.cfg，找到并编辑dataDir=D:\\zookeeper-3.4.10\\data

  4. windows的话，在cmd下运行：D:\zookeeper-3.4.13\bin>zkServer.cmd

D:\zookeeper-3.4.13\bin>zkServer.cmd

D:\zookeeper-3.4.13\bin>call "C:\Program Files\Java\jdk1.8.0_181"\bin\java "-Dzookeeper.log.dir=D:\zookeeper-3.4.13\bin\.." "-Dzookeeper.root.logger=INFO,CONSOLE" -cp "D:\zookeeper-3.4.13\bin\..\build\classes;D:\zookeeper-3.4.13\bin\..\build\lib\*;D:\zookeeper-3.4.13\bin\..\*;D:\zookeeper-3.4.13\bin\..\lib\*;D:\zookeeper-3.4.13\bin\..\conf" org.apache.zookeeper.server.quorum.QuorumPeerMain "D:\zookeeper-3.4.13\bin\..\conf\zoo.cfg"

2019-03-18 22:07:46,267 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: D:\zookeeper-3.4.13\bin\..\conf\zoo.cfg

2019-03-18 22:07:46,276 [myid:] - INFO  [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3

2019-03-18 22:07:46,276 [myid:] - INFO  [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0

2019-03-18 22:07:46,277 [myid:] - INFO  [main:DatadirCleanupManager@101] - Purge task is not scheduled.

2019-03-18 22:07:46,280 [myid:] - WARN  [main:QuorumPeerMain@116] - Either no config or no quorum defined in config, running  in standalone mode

2019-03-18 22:07:46,397 [myid:] - INFO  [main:QuorumPeerConfig@136] - Reading configuration from: D:\zookeeper-3.4.13\bin\..\conf\zoo.cfg

2019-03-18 22:07:46,398 [myid:] - INFO  [main:ZooKeeperServerMain@98] - Starting server

2019-03-18 22:07:46,422 [myid:] - INFO  [main:Environment@100] - Server environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT

2019-03-18 22:07:46,422 [myid:] - INFO  [main:Environment@100] - Server environment:host.name=DESKTOP-D5MA1P9

2019-03-18 22:07:46,422 [myid:] - INFO  [main:Environment@100] - Server environment:java.version=1.8.0_181

2019-03-18 22:07:46,423 [myid:] - INFO  [main:Environment@100] - Server environment:java.vendor=Oracle Corporation

2019-03-18 22:07:46,423 [myid:] - INFO  [main:Environment@100] - Server environment:java.home=C:\Program Files\Java\jdk1.8.0_181\jre

2019-03-18 22:07:46,423 [myid:] - INFO  [main:Environment@100] - Server environment:java.class.path=D:\zookeeper-3.4.13\bin\..\build\classes;D:\zookeeper-3.4.13\bin\..\build\lib\*;D:\zookeeper-3.4.13\bin\..\zookeeper-3.4.13.jar;D:\zookeeper-3.4.13\bin\..\lib\audience-annotations-0.5.0.jar;D:\zookeeper-3.4.13\bin\..\lib\jline-0.9.94.jar;D:\zookeeper-3.4.13\bin\..\lib\log4j-1.2.17.jar;D:\zookeeper-3.4.13\bin\..\lib\netty-3.10.6.Final.jar;D:\zookeeper-3.4.13\bin\..\lib\slf4j-api-1.7.25.jar;D:\zookeeper-3.4.13\bin\..\lib\slf4j-log4j12-1.7.25.jar;D:\zookeeper-3.4.13\bin\..\conf

2019-03-18 22:07:46,424 [myid:] - INFO  [main:Environment@100] - Server environment:java.library.path=C:\Program Files\Java\jdk1.8.0_181\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\nodejs\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Program Files\Java\jre1.8.0_191\bin;C:\Program Files\Java\jdk1.8.0_181\bin;C:\Users\yzd\AppData\Local\Microsoft\WindowsApps;C:\Users\yzd\AppData\Roaming\npm;.

2019-03-18 22:07:46,425 [myid:] - INFO  [main:Environment@100] - Server environment:java.io.tmpdir=C:\Users\yzd\AppData\Local\Temp\

2019-03-18 22:07:46,425 [myid:] - INFO  [main:Environment@100] - Server environment:java.compiler=<NA>

2019-03-18 22:07:46,427 [myid:] - INFO  [main:Environment@100] - Server environment:os.name=Windows 10

2019-03-18 22:07:46,427 [myid:] - INFO  [main:Environment@100] - Server environment:os.arch=amd64

2019-03-18 22:07:46,427 [myid:] - INFO  [main:Environment@100] - Server environment:os.version=10.0

2019-03-18 22:07:46,428 [myid:] - INFO  [main:Environment@100] - Server environment:user.name=yzd

2019-03-18 22:07:46,428 [myid:] - INFO  [main:Environment@100] - Server environment:user.home=C:\Users\yzd

2019-03-18 22:07:46,428 [myid:] - INFO  [main:Environment@100] - Server environment:user.dir=D:\zookeeper-3.4.13\bin

2019-03-18 22:07:46,438 [myid:] - INFO  [main:ZooKeeperServer@836] - tickTime set to 2000

2019-03-18 22:07:46,438 [myid:] - INFO  [main:ZooKeeperServer@845] - minSessionTimeout set to -1

2019-03-18 22:07:46,439 [myid:] - INFO  [main:ZooKeeperServer@854] - maxSessionTimeout set to -1

2019-03-18 22:07:46,647 [myid:] - INFO  [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory

2019-03-18 22:07:46,651 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181

搭建成功

zookeeper和eureka比较

ZooKeeper、Eureka对比：

https://www.cnblogs.com/jieqing/p/8394001.html

CAP定理（C-数据一致性；A-服务可用性；P-服务对网络分区故障的容错性，这三个特性在任何分布式系统中不能同时满足，最多同时满足两个）

ZooKeeper基于CP，不保证高可用，如果zookeeper正在选主，或者Zookeeper集群中半数以上机器不可用，那么将无法获得数据。Eureka基于AP，能保证高可用，即使所有机器都挂了，也能拿到本地缓存的数据。作为注册中心，其实配置是不经常变动的，只有发版和机器出故障时会变。对于不经常变动的配置来说，CP是不合适的，而AP在遇到问题时可以用牺牲一致性来保证可用性，既返回旧数据，缓存数据。

所以理论上Eureka是更适合作注册中心。而现实环境中大部分项目可能会使用ZooKeeper，那是因为集群不够大，并且基本不会遇到用做注册中心的机器一半以上都挂了的情况。所以实际上也没什么大问题。

kfaka

Kafka is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies.

Kafka是由Apache软件基金会开发的一个开源流处理平台，由Scala和Java编写。Kafka是一种高吞吐量的分布式发布订阅消息系统，它可以处理消费者规模的网站中的所有动作流数据。 这种动作（网页浏览，搜索和其他用户的行动）是在现代网络上的许多社会功能的一个关键因素。 这些数据通常是由于吞吐量的要求而通过处理日志和日志聚合来解决。 对于像Hadoop一样的日志数据和离线分析系统，但又要求实时处理的限制，这是一个可行的解决方案。Kafka的目的是通过Hadoop的并行加载机制来统一线上和离线的消息处理，也是为了通过集群来提供实时的消息。

kafka包内自带整合好的zookeeper，所以如果要搭建kafka可以省去单独下载zookeeper的步骤。想起了欲练武功必先自宫的笑话，哈哈哈。

环境搭建

  1. 下载地址：http://kafka.apache.org/downloads.html
  2. 解压缩，建立一个空文件夹 logs. eg: D:\kafka_2.11-1.0.0\logs
  3. 进入config目录，编辑 server.properties文件(eg: 用“写字板”打开)。找到并编辑log.dirs= D:\kafka_2.11-1.0.0\logs
  4. 请确保在启动Kafka服务器前，Zookeeper实例已经准备好并开始运行。
  5. 在此处打开Powershell窗口，运行：.\bin\windows\kafka-server-start.bat .\config\server.properties

PS D:\kafka_2.12-2.1.1> .\bin\windows\kafka-server-start.bat .\config\server.properties

[2019-03-18 22:15:34,866] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)

[2019-03-18 22:15:35,503] INFO starting (kafka.server.KafkaServer)

[2019-03-18 22:15:35,505] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)

[2019-03-18 22:15:35,530] INFO [ZooKeeperClient] Initializing a new session to localhost:2181. (kafka.zookeeper.ZooKeeperClient)

[2019-03-18 22:15:35,536] INFO Client environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 00:39 GMT (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,536] INFO Client environment:host.name=DESKTOP-D5MA1P9 (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,536] INFO Client environment:java.version=1.8.0_181 (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,537] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,537] INFO Client environment:java.home=C:\Program Files\Java\jdk1.8.0_181\jre (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,537] INFO Client environment:java.class.path=D:\kafka_2.12-2.1.1\libs\activation-1.1.1.jar;D:\kafka_2.12-2.1.1\libs\aopalliance-repackaged-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\argparse4j-0.7.0.jar;D:\kafka_2.12-2.1.1\libs\audience-annotations-0.5.0.jar;D:\kafka_2.12-2.1.1\libs\commons-lang3-3.8.1.jar;D:\kafka_2.12-2.1.1\libs\connect-api-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-basic-auth-extension-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-file-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-json-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-runtime-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\connect-transforms-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\guava-20.0.jar;D:\kafka_2.12-2.1.1\libs\hk2-api-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\hk2-locator-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\hk2-utils-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\jackson-annotations-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-core-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-databind-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-jaxrs-base-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-jaxrs-json-provider-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\jackson-module-jaxb-annotations-2.9.8.jar;D:\kafka_2.12-2.1.1\libs\javassist-3.22.0-CR2.jar;D:\kafka_2.12-2.1.1\libs\javax.annotation-api-1.2.jar;D:\kafka_2.12-2.1.1\libs\javax.inject-1.jar;D:\kafka_2.12-2.1.1\libs\javax.inject-2.5.0-b42.jar;D:\kafka_2.12-2.1.1\libs\javax.servlet-api-3.1.0.jar;D:\kafka_2.12-2.1.1\libs\javax.ws.rs-api-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\javax.ws.rs-api-2.1.jar;D:\kafka_2.12-2.1.1\libs\jaxb-api-2.3.0.jar;D:\kafka_2.12-2.1.1\libs\jersey-client-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-common-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-container-servlet-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-container-servlet-core-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-hk2-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-media-jaxb-2.27.jar;D:\kafka_2.12-2.1.1\libs\jersey-server-2.27.jar;D:\kafka_2.12-2.1.1\libs\jetty-client-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-continuation-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-http-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-io-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-security-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-server-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-servlet-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-servlets-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jetty-util-9.4.12.v20180830.jar;D:\kafka_2.12-2.1.1\libs\jopt-simple-5.0.4.jar;D:\kafka_2.12-2.1.1\libs\kafka-clients-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-log4j-appender-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-streams-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-streams-examples-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-streams-scala_2.12-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-streams-test-utils-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka-tools-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-javadoc.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-javadoc.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-scaladoc.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-scaladoc.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-sources.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-sources.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-test-sources.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-test-sources.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-test.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1-test.jar.asc;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1.jar;D:\kafka_2.12-2.1.1\libs\kafka_2.12-2.1.1.jar.asc;D:\kafka_2.12-2.1.1\libs\log4j-1.2.17.jar;D:\kafka_2.12-2.1.1\libs\lz4-java-1.5.0.jar;D:\kafka_2.12-2.1.1\libs\maven-artifact-3.6.0.jar;D:\kafka_2.12-2.1.1\libs\metrics-core-2.2.0.jar;D:\kafka_2.12-2.1.1\libs\osgi-resource-locator-1.0.1.jar;D:\kafka_2.12-2.1.1\libs\plexus-utils-3.1.0.jar;D:\kafka_2.12-2.1.1\libs\reflections-0.9.11.jar;D:\kafka_2.12-2.1.1\libs\rocksdbjni-5.14.2.jar;D:\kafka_2.12-2.1.1\libs\scala-library-2.12.7.jar;D:\kafka_2.12-2.1.1\libs\scala-logging_2.12-3.9.0.jar;D:\kafka_2.12-2.1.1\libs\scala-reflect-2.12.7.jar;D:\kafka_2.12-2.1.1\libs\slf4j-api-1.7.25.jar;D:\kafka_2.12-2.1.1\libs\slf4j-log4j12-1.7.25.jar;D:\kafka_2.12-2.1.1\libs\snappy-java-1.1.7.2.jar;D:\kafka_2.12-2.1.1\libs\validation-api-1.1.0.Final.jar;D:\kafka_2.12-2.1.1\libs\zkclient-0.11.jar;D:\kafka_2.12-2.1.1\libs\zookeeper-3.4.13.jar;D:\kafka_2.12-2.1.1\libs\zstd-jni-1.3.7-1.jar (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,538] INFO Client environment:java.library.path=C:\Program Files\Java\jdk1.8.0_181\bin;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\nodejs\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\Git\cmd;C:\Program Files\Java\jre1.8.0_191\bin;C:\Program Files\Java\jdk1.8.0_181\bin;C:\Users\yzd\AppData\Local\Microsoft\WindowsApps;C:\Users\yzd\AppData\Roaming\npm;. (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,539] INFO Client environment:java.io.tmpdir=C:\Users\yzd\AppData\Local\Temp\ (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,542] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,542] INFO Client environment:os.name=Windows 10 (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,543] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,543] INFO Client environment:os.version=10.0 (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,547] INFO Client environment:user.name=yzd (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,547] INFO Client environment:user.home=C:\Users\yzd (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,547] INFO Client environment:user.dir=D:\kafka_2.12-2.1.1 (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,549] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@3e2e18f2 (org.apache.zookeeper.ZooKeeper)

[2019-03-18 22:15:35,570] INFO [ZooKeeperClient] Waiting until connected. (kafka.zookeeper.ZooKeeperClient)

[2019-03-18 22:15:35,570] INFO Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)

[2019-03-18 22:15:35,574] INFO Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn)

[2019-03-18 22:15:35,598] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x1000ae3e3690000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)

[2019-03-18 22:15:35,602] INFO [ZooKeeperClient] Connected. (kafka.zookeeper.ZooKeeperClient)

[2019-03-18 22:15:35,974] INFO Cluster ID = brvhJ2K3R4qZnGJ6CQhghA (kafka.server.KafkaServer)

[2019-03-18 22:15:35,980] WARN No meta.properties file under dir D:\kafka_2.12-2.1.1\logs\meta.properties (kafka.server.BrokerMetadataCheckpoint)

[2019-03-18 22:15:36,049] INFO KafkaConfig values:

        advertised.host.name = null

        ...(省略若干)

        zookeeper.sync.time.ms = 2000

 (kafka.server.KafkaConfig)

[2019-03-18 22:15:36,092] INFO [ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)

[2019-03-18 22:15:36,092] INFO [ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)

[2019-03-18 22:15:36,094] INFO [ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)

[2019-03-18 22:15:36,132] INFO Loading logs. (kafka.log.LogManager)

[2019-03-18 22:15:36,143] INFO Logs loading complete in 11 ms. (kafka.log.LogManager)

[2019-03-18 22:15:36,161] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)

[2019-03-18 22:15:36,166] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)

[2019-03-18 22:15:36,486] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)

[2019-03-18 22:15:36,534] INFO [SocketServer brokerId=0] Started 1 acceptor threads (kafka.network.SocketServer)

[2019-03-18 22:15:36,562] INFO [ExpirationReaper-0-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2019-03-18 22:15:36,563] INFO [ExpirationReaper-0-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2019-03-18 22:15:36,563] INFO [ExpirationReaper-0-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2019-03-18 22:15:36,581] INFO [LogDirFailureHandler]: Starting (kafka.server.ReplicaManager$LogDirFailureHandler)

[2019-03-18 22:15:36,607] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.zk.KafkaZkClient)

[2019-03-18 22:15:36,612] INFO Result of znode creation at /brokers/ids/0 is: OK (kafka.zk.KafkaZkClient)

[2019-03-18 22:15:36,613] INFO Registered broker 0 at path /brokers/ids/0 with addresses: ArrayBuffer(EndPoint(DESKTOP-D5MA1P9,9092,ListenerName(PLAINTEXT),PLAINTEXT)) (kafka.zk.KafkaZkClient)

[2019-03-18 22:15:36,615] WARN No meta.properties file under dir D:\kafka_2.12-2.1.1\logs\meta.properties (kafka.server.BrokerMetadataCheckpoint)

[2019-03-18 22:15:36,682] INFO [ExpirationReaper-0-topic]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2019-03-18 22:15:36,684] INFO [ExpirationReaper-0-Heartbeat]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2019-03-18 22:15:36,685] INFO [ExpirationReaper-0-Rebalance]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

[2019-03-18 22:15:36,689] INFO Successfully created /controller_epoch with initial epoch 0 (kafka.zk.KafkaZkClient)

[2019-03-18 22:15:36,714] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.group.GroupCoordinator)

[2019-03-18 22:15:36,716] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.group.GroupCoordinator)

[2019-03-18 22:15:36,731] INFO [GroupMetadataManager brokerId=0] Removed 0 expired offsets in 15 milliseconds. (kafka.coordinator.group.GroupMetadataManager)

[2019-03-18 22:15:36,748] INFO [ProducerId Manager 0]: Acquired new producerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing to Zk with path version 1 (kafka.coordinator.transaction.ProducerIdManager)

[2019-03-18 22:15:36,780] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)

[2019-03-18 22:15:36,782] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)

[2019-03-18 22:15:36,791] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)

[2019-03-18 22:15:36,875] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)

[2019-03-18 22:15:36,892] INFO [SocketServer brokerId=0] Started processors for 1 acceptors (kafka.network.SocketServer)

[2019-03-18 22:15:36,901] INFO Kafka version : 2.1.1 (org.apache.kafka.common.utils.AppInfoParser)

[2019-03-18 22:15:36,919] INFO Kafka commitId : 21234bee31165527 (org.apache.kafka.common.utils.AppInfoParser)

[2019-03-18 22:15:36,922] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

搭建成功。

测试kafka生产者消费者

  1. 开启zookeeper和kafka服务

  2. 创建topic：

> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

看一眼刚才创建的主题

> bin/kafka-topics.sh --list --zookeeper localhost:2181

test
  1. 发消息：接收者
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

test

  1. 收消息：消费者
> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

This is a message

This is another message

参考

https://blog.csdn.net/tianmanchn/article/details/78943147

https://www.jianshu.com/p/ca60bc8c5673

关注我

我是蛮三刀把刀，硕士，后端开发工程师。主要关注后端开发，数据安全，爬虫等方向。微信：yangzd1102

Github个人主页：

https://github.com/qqxx6661

原创博客主要内容

  • Java知识点复习全手册
  • Leetcode算法题解析
  • 剑指offer算法题解析
  • Python爬虫相关技术实战
  • 后端开发相关技术实战
  • SpringCloud实战

个人公众号：后端技术漫谈

如果文章对你有帮助，不妨收藏起来并转发给您的朋友们~

快速搭建Zookeeper和Kafka环境的更多相关文章

  1. Docker快速搭建Zookeeper和kafka集群

    使用Docker快速搭建Zookeeper和kafka集群 镜像选择 Zookeeper和Kafka集群分别运行在不同的容器中zookeeper官方镜像,版本3.4kafka采用wurstmeiste ...

  2. 使用Docker快速搭建Zookeeper和kafka集群

    使用Docker快速搭建Zookeeper和kafka集群 镜像选择 Zookeeper和Kafka集群分别运行在不同的容器中zookeeper官方镜像,版本3.4kafka采用wurstmeiste ...

  3. 快速搭建 Node.js 开发环境以及加速 npm

    如何快速搭建 node 开发环境 npm 超慢 github 无法打开的问题 于是我觉得应该写一篇文章解答所有这些起步问题,让新同学也能顺顺利利入门. 快速搭建 Node.js 开发环境 如果你想长期 ...

  4. windows下9款一键快速搭建PHP本地运行环境的好工具(含php7.0环境)

    推荐几款一键快速搭建PHP本地运行环境的好工具(含php7.0及apache,nigix,mysql) 首推phpstudy2016和wampServer3.0.6     理由支持php7.0 目前 ...

  5. 【转】使用nvm快速搭建 Node.js 开发环境

    原文链接:http://www.cnblogs.com/shuoer/p/7802891.html 快速搭建 Node.js 开发环境 如果你想长期做 node 开发, 或者想快速更新 node 版本 ...

  6. 搭建zookeeper和Kafka集群

    搭建zookeeper和Kafka集群: 本实验拥有3个节点,均为CentOS 7系统,分别对应IP为10.211.55.11.10.211.55.13.10.211.55.14,且均有相同用户名 ( ...

  7. docker-compose搭建zookeeper集群环境 CodingCode

    docker-compose搭建zookeeper集群环境 使用docker-compose搭建zookeeper集群环境 zookeeper是一个集群环境,用来管理微服务架构下面的配置管理功能. 这 ...

  8. centos7 快速搭建redis集群环境

    本文主要是记录一下快速搭建redis集群环境的方式. 环境简介:centos 7  + redis-3.2.4 本次用两个服务6个节点来搭建:192.168.116.120  和  192.168.1 ...

  9. 快速搭建appium自动测试环境

    首先申明本文是基本于Python与Android来快速搭建Appium自动化测试环境: 主要分为以下几个步骤: 前提条件: 1)安装与配置python环境,打开 Python官网,找到"Do ...

  10. Linux快速搭建C/C++开发环境

    导读:越来越多的程序员在Linux下进行C/C++的开发.本文以CentOS 7为例,教你快速搭建一个vi + gcc/g++ + Make + valgrind的开发环境. 本文字数:1500,阅读 ...

随机推荐

  1. 在 .NET 中使用 OPC UA 协议

    目录 什么是 OPC UA UaExpert 的使用 下载 UaExpert 首次启动 添加 OPC UA 服务器 连接 OPC UA 服务器 查看 PLC 数据 使用 C# 读写 OPC UA 数据 ...

  2. Springboot访问html页面

    项目结构如图 1.html页面创建 在原有的项目resouces目录下创建static包,并在static下创建pages,然后在pages包下index.html. index.html内容 < ...

  3. mysql where子句 区分 大小写

    解决办法 使用binary关键字,此时就能区分大小写了 SELECT * FROM `tb_test` WHERE BINARY COL_1='abc'

  4. verilog之random

    verilog之random 1.基本作用 random,用于产生随机数.在测试时,有时需要测试的情况太多,无法一一列举,就需要使用抽样测试的方法验证功能是否可行.random是一个有返回值的系统函数 ...

  5. verilog之锁存器和触发器

    verilog锁存器和触发器 1.基本概念 锁存,就是输入信号变化时,输出不发生变化时,就是触发器或者锁存器.触发器的敏感信号是clk,即触发器是知道被延时了多少.对于锁存器来说,延时是不确定的.一般 ...

  6. Generalized Focal Loss：Focal loss魔改以及预测框概率分布，保涨点 | NeurIPS 2020

    为了高效地学习准确的预测框及其分布,论文对Focal loss进行拓展,提出了能够优化连续值目标的Generalized Focal loss,包含Quality Focal loss和Distrib ...

  7. 基于rv1126 rkmeida 一路多出 原理

    基于rv1126 rkmeida 一路多出的坑 首先说要的是介绍一下rkmedia 相关内容 ​ RKMedia提供了一种媒体处理方案,可支持应用软件快速开发.RKMedia在各模块基础API上做进一 ...

  8. #贪心，二叉堆#洛谷 1954 [NOI2010] 航空管制

    题目 分析 首先考虑可行方案,很容易想到拓扑排序, 但是如果建正图第一类的限制有可能不能满足, 考虑第一类限制其实时间倒流就是在 \(T\) 时刻之后才能选它. 那么直接建反图然后 \(a_i\) 大 ...

  9. WPF入门-导航页

    本文适合有winform或web前端基础,且有C#开发基础的同学学习. 本文介绍的开发模式属于MVVM,即Model-View-ViewModel .这种开发模式上手简单,开发迅速. wpf入门第一篇 ...

  10. Python将依赖包导出到requirements.txt文件

    代码 # 查询环境中已经安装的库 pip list # 将所有依赖库导出到 requirements.txt 文件 pip freeze > requirements.txt