问题:

1. kafka的topic 是程序自己建立,默认只建立8个partitions,1个replication-factor

目的: 扩展partitions 到9个, replicatoion-factor  3个

原因:因为kafka是三台,如果replicatoion-factor设置1的话,kafka坏掉一个会有影响

(1). 扩展partitions 到9个

/kafka-topics.sh --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka --alter --topic Message --partitions 9
 
查看结果:
/kafka-topics.sh --describe  --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka  --topic Message 

Topic:Message  PartitionCount:9 ReplicationFactor:1 Configs:
Topic: Message  Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic:Message  Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Topic: Message  Partition: 2 Leader: 3 Replicas: 3 Isr: 3
Topic: Message  Partition: 3 Leader: 1 Replicas: 1 Isr: 1
Topic: Message Partition: 4 Leader: 2 Replicas: 2 Isr: 2
Topic: Message  Partition: 5 Leader: 3 Replicas: 3 Isr: 3
Topic: Message Partition: 6 Leader: 1 Replicas: 1 Isr: 1
Topic: Message  Partition: 7 Leader: 2 Replicas: 2 Isr: 2

Topic: Message  Partition: 8 Leader: 3 Replicas: 3 Isr: 3

(2)扩展ReplicationFactor 为3

vim increace-factor.json

{"version":1,
"partitions":[
{"topic":"Message","partition":0,"replicas":[1,2,3]},
{"topic":"Message","partition":1,"replicas":[1,2,3]},
{"topic":"Message","partition":2,"replicas":[1,2,3]},
{"topic":"Message","partition":3,"replicas":[1,2,3]},
{"topic":"Message","partition":4,"replicas":[1,2,3]},
{"topic":"Message","partition":5,"replicas":[1,2,3]},
{"topic":"Message","partition":6,"replicas":[1,2,3]},
{"topic":"Message","partition":7,"replicas":[1,2,3]}

{"topic":"Message","partition":8,"replicas":[1,2,3]}
]}

执行如下命令:

/kafka-reassign-partitions.sh --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka --reassignment-json-file increace-factor.json --execute

[root@java]#/kafka-topics.sh --describe --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka  --topic Message
Topic:Message PartitionCount:9 ReplicationFactor:3 Configs:
Topic: Message Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: Message Partition: 1 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message Partition: 2 Leader: 1 Replicas: 1,2,3 Isr: 3,1,2
Topic: Message Partition: 3 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: Message Partition: 4 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message  Partition: 5 Leader: 1 Replicas: 1,2,3 Isr: 3,1,2
Topic: Message Partition: 6 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: cMessage Partition: 7 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3

Topic: Message  Partition: 8 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3

kafka 扩展partition和replication-factor的更多相关文章

  1. Error while executing topic command : Replication factor: 2 larger than available brokers: 0.

    [root@hdp1 /mnt/software/maxwell-1.19.4]#kafka-topics.sh --zookeeper hdp1,hdp2,hdp3:2181 --create -- ...

  2. Apache Kafka(十)Partitions与Replication Factor 调整准则

    Partitions与Replication Factor调整准则 Partition 数目与Replication Factor是在创建一个topic时非常重要的两个参数,这两个参数的取值会直接影响 ...

  3. kafka创建会话,报Error while executing topic command : Replication factor: 1 larger than available brokers: 0.

     bin/kafka-topics.sh --create --zookeeper es1:2181 --replication-factor 1 --partitions 1 --topic top ...

  4. 【kafka】kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0

    https://blog.csdn.net/bigtree_3721/article/details/78442912 I am trying to create topics in Kafka by ...

  5. kafka之partition分区及副本replica升级

    修改kafka的partition分区 bin/kafka-topics.sh --zookeeper datacollect-2:2181 --alter --partitions 3 --topi ...

  6. Kafka Topic Partition Replica Assignment实现原理及资源隔离方案

    本文共分为三个部分:   Kafka Topic创建方式 Kafka Topic Partitions Assignment实现原理 Kafka资源隔离方案   1. Kafka Topic创建方式 ...

  7. replication factor

    http://www.tuicool.com/articles/RJbIBj 关于Hadoop中replication factor解惑 时间 2014-06-09 08:00:50   ITeye ...

  8. Kafka Cached zkVersion [62] not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition) 问题分析

    我司业务Kafka集群是3节点(broker分别为10,20,30),每个Topic 3 Partition,3 Repilication的配置,早上起床突然发现所有Topic的Broker节点都变为 ...

  9. linux 下安装 php kafka 扩展

    我们使用官方推荐 php kafka 扩展 phpkafka,由于该扩展是基于 librdkafka 开发,所以我们首先需要安装 librdkafka 下载地址:http://kafka.apache ...

随机推荐

  1. hdu6393 /// 树链剖分

    题目大意: 给定n q 在n个点n条边的图中 进行q次操作 0 k x 为修改第k条边的值为x 1 x y 为查询x到y的最短路 https://blog.csdn.net/nka_kun/artic ...

  2. zabbix--监控的组件和进程介绍

    上图是zabbix的架构,zabbix proxy(代理),可以减小IO并发. zabbix web GUI是用php写的画图工具,从数据库抓取数据. zabbix database zabbix获取 ...

  3. 将时间 '2018-08-06T10:00:00.000Z' 格式转化为本地时间

    参考:https://blog.csdn.net/sxf_123456/article/details/81582964 参考模板: from datetime import datetime, ti ...

  4. DO_DEVICE_INITIALIZING

    这个东西的位置在DEVICE_OBJECT的Flags字段中, 本来这个Flags大多的情况下都是在设置IO方式,如DO_BUFFERED_IO, 但特殊的位也可能需要在这里设置. 用处是防止当自己的 ...

  5. shell脚本输出九九乘法表

    #!/bin/bash#输出九九乘法表 for ((i=1;i<=9;i++)) do for ((j=1;j<=$i;j++)) do echo -n $j'x'$i=$(($i*$j) ...

  6. Spring框架4大原则和主要功能

    Spring框架4大原则: 使用POJO进行轻量级和最小侵入式开发 POJO(Plain Ordinary Java Object)简单的Java对象,实际就是普通JavaBeans,是为了避免和EJ ...

  7. element-UI 点击一行,背景色变化

    代码: @row-click="rowClick" 当某一行被点击时会触发该事件 :row-class-name="tableRowClassName"  可以 ...

  8. ajax 工作原理

    Ajax的优缺点及工作原理? 定义和用法: AJAX = Asynchronous JavaScript and XML(异步的 JavaScript 和 XML).Ajax 是一种用于创建快速动态网 ...

  9. html input 上capture 参数在 安卓 苹果上的异同

    安卓上 <input type="file" accept="image/*" capture="camera"> //只调用相 ...

  10. 搜索进阶课件,视频,代码(状态压缩搜索,折半搜索,dfs,bfs总结)

    链接:https://pan.baidu.com/s/1-svffrprCOO4CtQoCTQ9hQ 提取码:h909 复制这段内容后打开百度网盘手机App,操作更方便哦