问题:

1. kafka的topic 是程序自己建立,默认只建立8个partitions,1个replication-factor

目的: 扩展partitions 到9个, replicatoion-factor  3个

原因:因为kafka是三台,如果replicatoion-factor设置1的话,kafka坏掉一个会有影响

(1). 扩展partitions 到9个

/kafka-topics.sh --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka --alter --topic Message --partitions 9
 
查看结果:
/kafka-topics.sh --describe  --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka  --topic Message 

Topic:Message  PartitionCount:9 ReplicationFactor:1 Configs:
Topic: Message  Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic:Message  Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Topic: Message  Partition: 2 Leader: 3 Replicas: 3 Isr: 3
Topic: Message  Partition: 3 Leader: 1 Replicas: 1 Isr: 1
Topic: Message Partition: 4 Leader: 2 Replicas: 2 Isr: 2
Topic: Message  Partition: 5 Leader: 3 Replicas: 3 Isr: 3
Topic: Message Partition: 6 Leader: 1 Replicas: 1 Isr: 1
Topic: Message  Partition: 7 Leader: 2 Replicas: 2 Isr: 2

Topic: Message  Partition: 8 Leader: 3 Replicas: 3 Isr: 3

(2)扩展ReplicationFactor 为3

vim increace-factor.json

{"version":1,
"partitions":[
{"topic":"Message","partition":0,"replicas":[1,2,3]},
{"topic":"Message","partition":1,"replicas":[1,2,3]},
{"topic":"Message","partition":2,"replicas":[1,2,3]},
{"topic":"Message","partition":3,"replicas":[1,2,3]},
{"topic":"Message","partition":4,"replicas":[1,2,3]},
{"topic":"Message","partition":5,"replicas":[1,2,3]},
{"topic":"Message","partition":6,"replicas":[1,2,3]},
{"topic":"Message","partition":7,"replicas":[1,2,3]}

{"topic":"Message","partition":8,"replicas":[1,2,3]}
]}

执行如下命令:

/kafka-reassign-partitions.sh --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka --reassignment-json-file increace-factor.json --execute

[root@java]#/kafka-topics.sh --describe --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka  --topic Message
Topic:Message PartitionCount:9 ReplicationFactor:3 Configs:
Topic: Message Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: Message Partition: 1 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message Partition: 2 Leader: 1 Replicas: 1,2,3 Isr: 3,1,2
Topic: Message Partition: 3 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: Message Partition: 4 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message  Partition: 5 Leader: 1 Replicas: 1,2,3 Isr: 3,1,2
Topic: Message Partition: 6 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: cMessage Partition: 7 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3

Topic: Message  Partition: 8 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3

kafka 扩展partition和replication-factor的更多相关文章

  1. Error while executing topic command : Replication factor: 2 larger than available brokers: 0.

    [root@hdp1 /mnt/software/maxwell-1.19.4]#kafka-topics.sh --zookeeper hdp1,hdp2,hdp3:2181 --create -- ...

  2. Apache Kafka(十)Partitions与Replication Factor 调整准则

    Partitions与Replication Factor调整准则 Partition 数目与Replication Factor是在创建一个topic时非常重要的两个参数,这两个参数的取值会直接影响 ...

  3. kafka创建会话,报Error while executing topic command : Replication factor: 1 larger than available brokers: 0.

     bin/kafka-topics.sh --create --zookeeper es1:2181 --replication-factor 1 --partitions 1 --topic top ...

  4. 【kafka】kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0

    https://blog.csdn.net/bigtree_3721/article/details/78442912 I am trying to create topics in Kafka by ...

  5. kafka之partition分区及副本replica升级

    修改kafka的partition分区 bin/kafka-topics.sh --zookeeper datacollect-2:2181 --alter --partitions 3 --topi ...

  6. Kafka Topic Partition Replica Assignment实现原理及资源隔离方案

    本文共分为三个部分:   Kafka Topic创建方式 Kafka Topic Partitions Assignment实现原理 Kafka资源隔离方案   1. Kafka Topic创建方式 ...

  7. replication factor

    http://www.tuicool.com/articles/RJbIBj 关于Hadoop中replication factor解惑 时间 2014-06-09 08:00:50   ITeye ...

  8. Kafka Cached zkVersion [62] not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition) 问题分析

    我司业务Kafka集群是3节点(broker分别为10,20,30),每个Topic 3 Partition,3 Repilication的配置,早上起床突然发现所有Topic的Broker节点都变为 ...

  9. linux 下安装 php kafka 扩展

    我们使用官方推荐 php kafka 扩展 phpkafka,由于该扩展是基于 librdkafka 开发,所以我们首先需要安装 librdkafka 下载地址:http://kafka.apache ...

随机推荐

  1. PE代码段中的数据

    PE代码段中可能包含一些数据,比如 optional header中的data directory会索引到一些数据,比如import/export table等等: 还有一些jump table/sw ...

  2. URAL 1748 The Most Complex Number

    题目链接:https://vjudge.net/problem/11177 题目大意: 求小于等于 n 的最大反素数. 分析: n <= 10^18,而前20个素数的乘积早超过10^18,因此可 ...

  3. DNF抽奖活动

    活动内容: DNF用户在注册页面注册获得抽奖资格(或分享好友注册)参与抽奖,产生奖品后,活动参与用户,在活动领奖页面领取奖品,金币及点券需填写相应游戏区服.qq号等信息,并且为防止活动刷子,在领取页提 ...

  4. hdu6395 /// 优先队列dijkstra

    题目大意: 给定无向图的n m为点数和边数 接下来m行给定u v id表示点u到点v间有一条编号为id的边 当由一条边走到另一条边 而两条边的编号不同时 费用+1 优先队列跑dijkstra最短路 按 ...

  5. Tomcat爆破

    把输入的账户和密码包起来 选择第三个模式 第一个添加用户名 第二个添加: 第三个添加密码 选择编码格式 取消打钩

  6. linux 定时执行sql

    说明: 放执行脚本的路径是: /home/vagrant/ssh 文件夹结构: /home |_ vagrant |__ ssh |___ move_order_old_data.sh |___ mo ...

  7. vue tabNav 点击

    <template> <div class="content"> <header class="tab_nav"> < ...

  8. P1487 失落的成绩单

    P1487 失落的成绩单a[i]=a[i-2]-2.0*a[i-1]+2.0*d;a[2]越大,a[3]越小a[3]越大,a[4]越小所以a[2]越大,a[4]越大,a[3]越小就有了单调性,分奇偶进 ...

  9. poi之Excel(在线生成)下载

    Apache POI是Apache软件基金会的开放源码函式库,POI提供API给Java程序对Microsoft Office格式档案读和写的功能. poi之Excel下载 @RequestMappi ...

  10. 微信小程序--跳转页面常用的两种方法

    一.bindtap="onProductsItemTap"绑定点击跳转事件 在.wxml文件中绑定 在.js文件中实现绑定事件函数 二.navigator标签配合URL跳转法 在w ...