kafka 扩展partition和replication-factor
问题:
1. kafka的topic 是程序自己建立,默认只建立8个partitions,1个replication-factor
目的: 扩展partitions 到9个, replicatoion-factor 3个
原因:因为kafka是三台,如果replicatoion-factor设置1的话,kafka坏掉一个会有影响
(1). 扩展partitions 到9个
Topic:Message PartitionCount:9 ReplicationFactor:1 Configs:
Topic: Message Partition: 0 Leader: 1 Replicas: 1 Isr: 1
Topic:Message Partition: 1 Leader: 2 Replicas: 2 Isr: 2
Topic: Message Partition: 2 Leader: 3 Replicas: 3 Isr: 3
Topic: Message Partition: 3 Leader: 1 Replicas: 1 Isr: 1
Topic: Message Partition: 4 Leader: 2 Replicas: 2 Isr: 2
Topic: Message Partition: 5 Leader: 3 Replicas: 3 Isr: 3
Topic: Message Partition: 6 Leader: 1 Replicas: 1 Isr: 1
Topic: Message Partition: 7 Leader: 2 Replicas: 2 Isr: 2
Topic: Message Partition: 8 Leader: 3 Replicas: 3 Isr: 3
(2)扩展ReplicationFactor 为3
vim increace-factor.json
{"version":1,
"partitions":[
{"topic":"Message","partition":0,"replicas":[1,2,3]},
{"topic":"Message","partition":1,"replicas":[1,2,3]},
{"topic":"Message","partition":2,"replicas":[1,2,3]},
{"topic":"Message","partition":3,"replicas":[1,2,3]},
{"topic":"Message","partition":4,"replicas":[1,2,3]},
{"topic":"Message","partition":5,"replicas":[1,2,3]},
{"topic":"Message","partition":6,"replicas":[1,2,3]},
{"topic":"Message","partition":7,"replicas":[1,2,3]}
{"topic":"Message","partition":8,"replicas":[1,2,3]}
]}
执行如下命令:
/kafka-reassign-partitions.sh --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka --reassignment-json-file increace-factor.json --execute
[root@java]#/kafka-topics.sh --describe --zookeeper kafka-zookeeper01:2181,kafka-zookeeper02:2181,kafka-zookeeper03:2181/kafka --topic Message
Topic:Message PartitionCount:9 ReplicationFactor:3 Configs:
Topic: Message Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: Message Partition: 1 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message Partition: 2 Leader: 1 Replicas: 1,2,3 Isr: 3,1,2
Topic: Message Partition: 3 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: Message Partition: 4 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message Partition: 5 Leader: 1 Replicas: 1,2,3 Isr: 3,1,2
Topic: Message Partition: 6 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: cMessage Partition: 7 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
Topic: Message Partition: 8 Leader: 1 Replicas: 1,2,3 Isr: 2,1,3
kafka 扩展partition和replication-factor的更多相关文章
- Error while executing topic command : Replication factor: 2 larger than available brokers: 0.
[root@hdp1 /mnt/software/maxwell-1.19.4]#kafka-topics.sh --zookeeper hdp1,hdp2,hdp3:2181 --create -- ...
- Apache Kafka(十)Partitions与Replication Factor 调整准则
Partitions与Replication Factor调整准则 Partition 数目与Replication Factor是在创建一个topic时非常重要的两个参数,这两个参数的取值会直接影响 ...
- kafka创建会话,报Error while executing topic command : Replication factor: 1 larger than available brokers: 0.
bin/kafka-topics.sh --create --zookeeper es1:2181 --replication-factor 1 --partitions 1 --topic top ...
- 【kafka】kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
https://blog.csdn.net/bigtree_3721/article/details/78442912 I am trying to create topics in Kafka by ...
- kafka之partition分区及副本replica升级
修改kafka的partition分区 bin/kafka-topics.sh --zookeeper datacollect-2:2181 --alter --partitions 3 --topi ...
- Kafka Topic Partition Replica Assignment实现原理及资源隔离方案
本文共分为三个部分: Kafka Topic创建方式 Kafka Topic Partitions Assignment实现原理 Kafka资源隔离方案 1. Kafka Topic创建方式 ...
- replication factor
http://www.tuicool.com/articles/RJbIBj 关于Hadoop中replication factor解惑 时间 2014-06-09 08:00:50 ITeye ...
- Kafka Cached zkVersion [62] not equal to that in zookeeper, skip updating ISR (kafka.cluster.Partition) 问题分析
我司业务Kafka集群是3节点(broker分别为10,20,30),每个Topic 3 Partition,3 Repilication的配置,早上起床突然发现所有Topic的Broker节点都变为 ...
- linux 下安装 php kafka 扩展
我们使用官方推荐 php kafka 扩展 phpkafka,由于该扩展是基于 librdkafka 开发,所以我们首先需要安装 librdkafka 下载地址:http://kafka.apache ...
随机推荐
- PE代码段中的数据
PE代码段中可能包含一些数据,比如 optional header中的data directory会索引到一些数据,比如import/export table等等: 还有一些jump table/sw ...
- URAL 1748 The Most Complex Number
题目链接:https://vjudge.net/problem/11177 题目大意: 求小于等于 n 的最大反素数. 分析: n <= 10^18,而前20个素数的乘积早超过10^18,因此可 ...
- DNF抽奖活动
活动内容: DNF用户在注册页面注册获得抽奖资格(或分享好友注册)参与抽奖,产生奖品后,活动参与用户,在活动领奖页面领取奖品,金币及点券需填写相应游戏区服.qq号等信息,并且为防止活动刷子,在领取页提 ...
- hdu6395 /// 优先队列dijkstra
题目大意: 给定无向图的n m为点数和边数 接下来m行给定u v id表示点u到点v间有一条编号为id的边 当由一条边走到另一条边 而两条边的编号不同时 费用+1 优先队列跑dijkstra最短路 按 ...
- Tomcat爆破
把输入的账户和密码包起来 选择第三个模式 第一个添加用户名 第二个添加: 第三个添加密码 选择编码格式 取消打钩
- linux 定时执行sql
说明: 放执行脚本的路径是: /home/vagrant/ssh 文件夹结构: /home |_ vagrant |__ ssh |___ move_order_old_data.sh |___ mo ...
- vue tabNav 点击
<template> <div class="content"> <header class="tab_nav"> < ...
- P1487 失落的成绩单
P1487 失落的成绩单a[i]=a[i-2]-2.0*a[i-1]+2.0*d;a[2]越大,a[3]越小a[3]越大,a[4]越小所以a[2]越大,a[4]越大,a[3]越小就有了单调性,分奇偶进 ...
- poi之Excel(在线生成)下载
Apache POI是Apache软件基金会的开放源码函式库,POI提供API给Java程序对Microsoft Office格式档案读和写的功能. poi之Excel下载 @RequestMappi ...
- 微信小程序--跳转页面常用的两种方法
一.bindtap="onProductsItemTap"绑定点击跳转事件 在.wxml文件中绑定 在.js文件中实现绑定事件函数 二.navigator标签配合URL跳转法 在w ...