Kafka自我学习1-Multi-broker cluster

====================================Testing environment ===========================================

# cat /etc/redhat-release
CentOS Linux release 7.5. (Core) 

====================================Setting up a single-node broker ===========================================
check monitor port
# ss -t -a

1.download kafka package
# wget http://mirrors.hust.edu.cn/apache/kafka/1.1.0/kafka_2.11-1.1.0.tgz

2.uncompress kafka
# tar xzf kafka_2.11-1.1.0.tgz

3.move kafka folder
# mv kafka_2.11-1.1.0 /usr/local/kafka

4.start zookeeper by default configuration
# /usr/local/kafka/bin/zookeeper-server-start.sh config/zookeeper.properties

5.start kafka by default configuration(broker id=0, port=9092)
# /usr/local/kafka/bin/kafka-server-start.sh config/server.properties

6.create test1 topic
# /usr/local/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test1

7.check test1 topic status
# /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper localhost:2181
Topic:test1    PartitionCount:1    ReplicationFactor:1    Configs:
    Topic: test1    Partition: 0    Leader: 0    Replicas: 0    Isr: 0

8.produce a message data to test1 topic
# /usr/local/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test1
>helloworld

9.consume the message data from test1 topic
# /usr/local/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic test1
>helloworld

==============================Setting up a multi-broker cluster on one server=========================================
1.create broker1(id=1,port=9093), broker2(id=2,port=9094)
# cd /usr/local/kafka/config/
# cp server.properties server1.properties
# cp server.properties server2.properties

2.start broker1 and broker2
# /usr/local/kafka/bin/kafka-server-start.sh config/server1.properties &
# /usr/local/kafka/bin/kafka-server-start.sh config/server2.properties &

3.create my-repliactied-topic topic with 3 replication times
# /usr/local/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic

4.check my-repliactied-topic topic status, and this is kafka cluster, broker0,broker1,broker2 have 3 copy
# /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic
Topic:my-replicated-topic    PartitionCount:1    ReplicationFactor:3    Configs:
    Topic: my-replicated-topic    Partition: 0    Leader: 2    Replicas: 2,0,1    Isr: 2,0,1

5.produce a message data to my-repliactied-topic topic
# /usr/local/kafka/bin/kafka-console-producer.sh --broker-list localhost:9094 --topic my-replicated-topic
>helloworld1234

6.consume the message data from my-repliactied-topic topic
# /usr/local/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9094 --from-beginning --topic my-replicated-topic
>helloworld1234

7.Search broker2 PID
# ps aux | grep server2.properties
7564 ttys002    0:15.91 /System/Library/Frameworks/JavaVM...................server2.properties

8.Kill broker2
# kill -9 7564

9.broker2 is dead, but can continue to produce another message data to my-repliactied-topic topic
# /usr/local/kafka/bin/kafka-console-producer.sh --broker-list localhost:9094 --topic my-replicated-topic
>helloworld1234
>56789

10.broker2 is dead, but can continue to consume the message data from my-repliactied-topic topic
# /usr/local/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9094 --from-beginning --topic my-replicated-topic
>helloworld1234
[2018-05-21 16:25:14,334] WARN [Consumer clientId=consumer-1, groupId=console-consumer-82232] Connection to node 2 could not be established. Broker may not be available.
>56789

11.check my-repliactied-topic topic status, Leader transfer to broker0, and broker2 stop sync
# /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic
Topic:my-replicated-topic    PartitionCount:1    ReplicationFactor:3    Configs:
    Topic: my-replicated-topic    Partition: 0    Leader: 0    Replicas: 2,0,1    Isr: 0,1

Refer: https://kafka.apache.org/quickstart