Redis集群模式之分布式集群模式

前言

Redis集群模式主要有2种:

主从集群
分布式集群。

前者主要是为了高可用或是读写分离，后者为了更好的存储数据，负载均衡。
本文主要讲解主从集群。本章主要讲解后一半部分，Redis集群。

与本文相关的代码与配置文件都已经上传至github上:
地址: https://github.com/SeanYanxml/bigdata

原理

Redis为了实现负载均衡，提供集群模式。以三个节点为例，集群模式相当于将1-15000片分片，分为1-5000、5000-10000、10000-15000。每个节点分一段数据片。这样的话，当一个节点宕机后，这个节点没有备份的话，此段分片将不再可以使用。所以，官方推荐，集群内的每个节点都应该配备一个从节点，作为冷备。部署原理图如下所示(暂略)。

部署

由于没有那么多的机器，所以我们一般单机部署6个节点(3主3从)，也就是官网推荐的模式。

主要步骤如下:

安装ruby，因为分配集群的代码时ruby脚本；
分配集群
查看集群结果

部署代码

# mkdir cluster-test

# cd cluster-test

# mkdir 7000 7001 7002 7003 7004 7005

# 在文件夹内 分别放置redis.conf文件 文件内容见下

# 启动6个节点

# nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

＃ 使用ruby脚本分配集群资源

./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

# redis.conf

port 7000

cluster-enabled yes

cluster-config-file nodes.conf

cluster-node-timeout 5000

appendonly yes

### 启动redis进程7000 7001 7002 7003 7004 7005

touch start-all.sh

###  内容如下

cd 7000

nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

cd ../7001

nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

cd ../7002

nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

cd ../7003

nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

cd ../7004

nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

cd ../7005

nohup ../../src/redis-server  redis.conf > start.log 2>&1 &

chmod +x start-all.sh

./start-all.sh

ps -ef|grep redis 查看进程

[cpic@cpic-redis2-77 src]$ ps -ef|grep redis

cpic     19876     1  0 3月21 ?       05:14:19 ./redis-server *:6379

cpic     19886     1  0 3月21 ?       06:17:36 ./redis-sentinel *:26379 [sentinel]

cpic     25971     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7000 [cluster]

cpic     25972     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7001 [cluster]

cpic     25973     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7002 [cluster]

cpic     25974     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7003 [cluster]

cpic     25975     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7004 [cluster]

cpic     25976     1  0 15:07 pts/0    00:00:00 ../../src/redis-server *:7005 [cluster]

cpic     26273 24815  0 15:15 pts/0    00:00:00 grep --color=auto redis

成功结果

＃ create 脚本初始化后脚本

[root@cpic-redis-76 src]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

127.0.0.1:7000

127.0.0.1:7001

127.0.0.1:7002

Adding replica 127.0.0.1:7003 to 127.0.0.1:7000

Adding replica 127.0.0.1:7004 to 127.0.0.1:7001

Adding replica 127.0.0.1:7005 to 127.0.0.1:7002

M: 562665aca5f7db25bbd81e7b971c2f4c9aa65f96 127.0.0.1:7000

   slots:0-5460 (5461 slots) master

M: 6f93b84f7da7d6ea1b77bce1709432de472bbbe3 127.0.0.1:7001

   slots:5461-10922 (5462 slots) master

M: 6522a9ddec5b40cade141793351df44fe40a529f 127.0.0.1:7002

   slots:10923-16383 (5461 slots) master

S: f122aea6aa02be27f77454c0fcde9c65b891cd47 127.0.0.1:7003

   replicates 562665aca5f7db25bbd81e7b971c2f4c9aa65f96

S: e5b5bbd378d8b80250dca6fb5db80e755f88c53c 127.0.0.1:7004

   replicates 6f93b84f7da7d6ea1b77bce1709432de472bbbe3

S: 93ac017741ad4e65a4de3cb17ae295e6e5505b9a 127.0.0.1:7005

   replicates 6522a9ddec5b40cade141793351df44fe40a529f

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join...

>>> Performing Cluster Check (using node 127.0.0.1:7000)

M: 562665aca5f7db25bbd81e7b971c2f4c9aa65f96 127.0.0.1:7000

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 6f93b84f7da7d6ea1b77bce1709432de472bbbe3 127.0.0.1:7001

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: e5b5bbd378d8b80250dca6fb5db80e755f88c53c 127.0.0.1:7004

   slots: (0 slots) slave

   replicates 6f93b84f7da7d6ea1b77bce1709432de472bbbe3

M: 6522a9ddec5b40cade141793351df44fe40a529f 127.0.0.1:7002

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 93ac017741ad4e65a4de3cb17ae295e6e5505b9a 127.0.0.1:7005

   slots: (0 slots) slave

   replicates 6522a9ddec5b40cade141793351df44fe40a529f

S: f122aea6aa02be27f77454c0fcde9c65b891cd47 127.0.0.1:7003

   slots: (0 slots) slave

   replicates 562665aca5f7db25bbd81e7b971c2f4c9aa65f96

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

# 自动生成的node.conf文件

[root@cpic-redis-76 7005]# cat nodes.conf

1c7cbff1069612078d63329e5841c7430757a2cd 127.0.0.1:7003@17003 slave f5cc2de688640de572d16d72c2f174de027609a6 0 1516672833000 4 connected

dd09a06827e9bfeab3876f4464cd467f6d1279f0 127.0.0.1:7002@17002 master - 0 1516672833086 3 connected 10923-16383

5863f9ee11d3ee3beef466d76f48f221b48aa6b5 127.0.0.1:7004@17004 slave 3ed6531ec985e50e87ab8436d49f4c32ce82bdef 0 1516672832000 5 connected

3ed6531ec985e50e87ab8436d49f4c32ce82bdef 127.0.0.1:7001@17001 master - 0 1516672832000 2 connected 5461-10922

9013209020c2fe2322df8c420b1c251fe9c84955 127.0.0.1:7005@17005 myself,slave dd09a06827e9bfeab3876f4464cd467f6d1279f0 0 1516672831000 6 connected

f5cc2de688640de572d16d72c2f174de027609a6 127.0.0.1:7000@17000 master - 0 1516672832885 1 connected 0-5460

vars currentEpoch 6 lastVoteEpoch 0

# 启动的log日志

[root@cpic-redis-76 7005]# cat start.log

nohup: 忽略输入

5627:C 22 Jan 21:02:58.212 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

5627:C 22 Jan 21:02:58.212 # Redis version=4.0.6, bits=64, commit=00000000, modified=0, pid=5627, just started

5627:C 22 Jan 21:02:58.212 # Configuration loaded

5627:M 22 Jan 21:02:58.214 * Increased maximum number of open files to 10032 (it was originally set to 1024).

5627:M 22 Jan 21:02:58.215 * Node configuration loaded, I'm 9013209020c2fe2322df8c420b1c251fe9c84955

5627:M 22 Jan 21:02:58.216 * Running mode=cluster, port=7005.

5627:M 22 Jan 21:02:58.216 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.

5627:M 22 Jan 21:02:58.216 # Server initialized

5627:M 22 Jan 21:02:58.216 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.

5627:M 22 Jan 21:02:58.216 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.

5627:M 22 Jan 21:02:58.216 * Ready to accept connections

5627:M 23 Jan 10:00:27.212 # configEpoch set to 6 via CLUSTER SET-CONFIG-EPOCH

5627:M 23 Jan 10:00:27.286 # IP address for this node updated to 127.0.0.1

5627:S 23 Jan 10:00:31.238 * Before turning into a slave, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.

5627:S 23 Jan 10:00:31.239 # Cluster state changed: ok

5627:S 23 Jan 10:00:31.981 * Connecting to MASTER 127.0.0.1:7002

5627:S 23 Jan 10:00:31.981 * MASTER <-> SLAVE sync started

5627:S 23 Jan 10:00:31.981 * Non blocking connect for SYNC fired the event.

5627:S 23 Jan 10:00:31.981 * Master replied to PING, replication can continue...

5627:S 23 Jan 10:00:31.982 * Trying a partial resynchronization (request 033127a1144abc3683a83a6b7257c2b56d2bd0b8:1).

5627:S 23 Jan 10:00:31.983 * Full resync from master: e5e51569d1f49762952e6cc05e1f13ddff68719e:0

5627:S 23 Jan 10:00:31.983 * Discarding previously cached master state.

5627:S 23 Jan 10:00:32.084 * MASTER <-> SLAVE sync: receiving 175 bytes from master

5627:S 23 Jan 10:00:32.084 * MASTER <-> SLAVE sync: Flushing old data

5627:S 23 Jan 10:00:32.084 * MASTER <-> SLAVE sync: Loading DB in memory

5627:S 23 Jan 10:00:32.084 * MASTER <-> SLAVE sync: Finished with success

5627:S 23 Jan 10:00:32.085 * Background append only file rewriting started by pid 10164

5627:S 23 Jan 10:00:32.108 * AOF rewrite child asks to stop sending diffs.

10164:C 23 Jan 10:00:32.108 * Parent agreed to stop sending diffs. Finalizing AOF...

10164:C 23 Jan 10:00:32.108 * Concatenating 0.00 MB of AOF diff received from parent.

10164:C 23 Jan 10:00:32.108 * SYNC append only file rewrite performed

10164:C 23 Jan 10:00:32.109 * AOF rewrite: 0 MB of memory used by copy-on-write

5627:S 23 Jan 10:00:32.181 * Background AOF rewrite terminated with success

5627:S 23 Jan 10:00:32.181 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)

5627:S 23 Jan 10:00:32.181 * Background AOF rewrite finished successfully

BUG

BUG1

[root@cpic-redis-76 src]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

>>> Creating cluster

[ERR] Node 127.0.0.1:7005 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

解决办法: 删掉原有数据dump.rdb/appendonly.aof/nodes.conf文件。

BUG2

ruby安装一些列困难。

详见Redis Cluster 运维环境安装记录。

Reference

见Redis Cluster 运维环境安装记录

Redis集群模式之分布式集群模式的更多相关文章

solr 集群（SolrCloud 分布式集群部署步骤）
SolrCloud 分布式集群部署步骤安装软件包准备 apache-tomcat-7.0.54 jdk1.7 solr-4.8.1 zookeeper-3.4.5 注:以上软件都是基于 Linux ...
Solr系列二：solr-部署详解（solr两种部署模式介绍、独立服务器模式详解、SolrCloud分布式集群模式详解）
一.solr两种部署模式介绍 Standalone Server 独立服务器模式:适用于数据规模不大的场景 SolrCloud 分布式集群模式:适用于数据规模大,高可靠.高可用.高并发的场景二.独 ...
Hadoop（三）手把手教你搭建Hadoop全分布式集群
前言上一篇介绍了伪分布式集群的搭建,其实在我们的生产环境中我们肯定不是使用只有一台服务器的伪分布式集群当中的.接下来我将给大家分享一下全分布式集群的搭建! 其实搭建最基本的全分布式集群和伪分布式集群 ...
Hadoop（三）搭建Hadoop全分布式集群
原文地址:http://www.cnblogs.com/zhangyinhua/p/7652686.html 阅读目录(Content) 一.搭建Hadoop全分布式集群前提 1.1.网络 1.2.安 ...
MinIO分布式集群的扩展方案及实现
目录一.命令行方式扩展 1. MinIO扩展集群支持的命令语法 2. 扩容示例二.etcd扩展方案 1. 环境变量 2. 运行多个集群 3. 示例相关链接 MinIO 支持两种扩展方式: 通过修 ...
分布式缓存技术redis学习系列（四）——redis高级应用（集群搭建、集群分区原理、集群操作）
本文是redis学习系列的第四篇,前面我们学习了redis的数据结构和一些高级特性,点击下面链接可回看 <详细讲解redis数据结构(内存模型)以及常用命令> <redis高级应用( ...
分布式缓存技术redis学习（四）——redis高级应用（集群搭建、集群分区原理、集群操作）
本文是redis学习系列的第四篇,前面我们学习了redis的数据结构和一些高级特性,点击下面链接可回看 <详细讲解redis数据结构(内存模型)以及常用命令> <redis高级应用( ...
分布式缓存技术redis系列（四）——redis高级应用（集群搭建、集群分区原理、集群操作）
本文是redis学习系列的第四篇,前面我们学习了redis的数据结构和一些高级特性,点击下面链接可回看 <详细讲解redis数据结构(内存模型)以及常用命令> <redis高级应用( ...
Redis面试题及分布式集群
Reference: http://blog.csdn.net/yajlv/article/details/73467865 1. 使用Redis有哪些好处? (1) 速度快,因为数据存在内存中,类似 ...

随机推荐

krpano 全景学习
krpano 切片工具下载 https://krpano.com/tools/ krpano 案例使用 https://krpano.com/examples/usage/#top krpano 是 ...
centos系统基本操作命令
系统相关命令查看系统版本: cat /etc/centos-release 系统更新: yum update 用户相关命令增加用户: useradd [用户名] 设置密码:password ...
MySql翻页查询
分页查询在网页中随处可见,那原理是什么呢?下面简单介绍一下基于MySql数据库的limit实现方法. 首先明确为什么要使用分页查询,因为数据庞大,查询不可能全部显示在页面上,如果全部显示在页面上,也会 ...
VBA输入框（InputBox）(六)
InputBox函数提示用户输入值.当输入值后,如果用户单击确定按钮或按下键盘上的ENTER 键,InputBox函数将返回文本框中的文本.如果用户单击“取消” 按钮,该函数将返回一个空字符串(&q ...
es和redis cluster高可用扩容等等
https://www.jianshu.com/p/ec465da21b4a https://www.cnblogs.com/hello-shf/p/11543468.html https://www ...
Java 面向对象（五）抽象
一.抽象概述 1.由来父类中的方法,被它的子类们重写,子类各自的实现都不尽相同.那么父类的方法声明和方法主体,只有声明还有意义,而方法主体则没有存在的意义了. 我们把没有方法主体的方法称为抽象方法. ...
swiper按钮点击无效及控制器无效问题
点击箭头图片切换的同时,下面小图标也会随着切换,同理下面小图标切换时,上面也随着滚动. 示例代码如下:  <div class="swiper ...
2.Java集合-ConcurrentHashMap实现原理及源码分析
一.为何用ConcurrentHashMap 在并发编程中使用HashMap可能会导致死循环,而使用线程安全的HashTable效率又低下. 线程不安全的HashMap 在多线程环境下,使用HashM ...
fcrackzip破解zip密码
kail系统里有fcrackzip工具,可以对设置密码的压缩包zip直接进行破解,不用字典,直接进行暴力破解. fcrackzip 所用到的参数介绍如下:-b 表示暴力破解-c 表示暴力破解中使用的字 ...
EditText编辑框
<?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android=&quo ...