Redis 学习之集群

该文使用centos6.5 64位 redis3.2.8

一、 redis-cluster架构图

集群通信：所有redis节点之间通过PING-PONG机制彼此互联，内部使用二进制鞋子优化传输速度和带宽。

集群数据存储（哈希槽）：

（1）redis-cluster把所有的物理节点映射到[0-16383]slot上,cluster 负责维护node<->slot<->value
（2）Redis 集群中内置了 16384 个哈希槽，当需要在 Redis 集群中放置一个 key-value 时，redis 先对 key 使用 crc16 算法算出一个结果，然后把结果对 16384 求余数，这样每个 key 都会对应一个编号在 0-16383 之间的哈希槽，redis 会根据节点数量大致均等的将哈希槽映射到不同的节点。

举例说明数据如何选择存储在哪台redis服务器上：

现在有3台redis服务器的集群：redis Server1(0-5000) redis Server1(5001-10000) redis Server1(10001-16384)

存储数据：set key A

A存储在哪个节点上：

计算A的hash值，例如值为98，98这个槽在redis Server1上，所以A应该放到redis Server1。

节点之间如何确定某个节点挂掉:容错机制

判断节点是否挂掉：某个节点（黄色节点）给红色节点发送PING命令，但未得带PONG回应，于是该节点怀疑红色节点已经挂掉，于是他把这个猜想通知给其他节点，其他节点就会向红色节点发送PING命令，如果没有PONG回应，则也认为红色节点已经挂掉，如果半数以上master节点与master节点通信超过(cluster-node-timeout),认为当前master节点挂掉。

什么时候整个集群不可用(cluster_state:fail):

a:如果集群任意master挂掉,且当前master没有slave.集群进入fail状态,也可以理解成集群的slot映射[0-16383]不完整时进入fail状态，所以一般我们会为每一个master节点做从节点slave 。ps : redis-3.0.0.rc1加入cluster-require-full-coverage参数,默认关闭,打开集群兼容部分失败.

b:如果集群超过半数以上master挂掉，无论是否有slave,集群进入fail状态.

ps:当集群不可用时,所有对集群的操作做都不可用，收到((error) CLUSTERDOWN The cluster is down)错误

二、搭建redis集群

集群中有三个节点的集群，每个节点有一主一备。需要6台虚拟机。

在这里我们搭建一个伪分布式的集群，使用6个redis实例来模拟。

1、集群需要ruby环境：

安装ruby

yum install ruby

yum install rubygems

2.查看redis集群管理工具redis-trib.rb是否存在

[root@localhost src]# pwd

/tools/redis-3.2.8/src

[root@localhost src]# ll *.rb

-rwxrwxr-x. 1 root root 60852 2月  12 23:14 redis-trib.rb

[root@localhost src]#

3、安装redis-trib.rb脚本运行依赖的环境 redis-3.2.2.gem

   安装ruby的包：gem install redis-3.2.2.gem

4、创建6个redis实例

[root@localhost local]# pwd

/usr/local

[root@localhost local]# mkdir redis-cluster

[root@localhost local]# cp -r redis redis-cluster/redis01  //复制我们安装的单机版redis实例

//删除复制后的 dump.rdb 文件

[root@localhost bin]# pwd
/usr/local/redis-cluster/redis01/bin

[root@localhost bin]# rm -rf dump.rdb

//修改配置文件 port    与 cluster-enabled

# Accept connections on the specified port, default is 6379 (IANA #815344).

# If port 0 is specified Redis will not listen on a TCP socket.

port 7001

################################ REDIS CLUSTER  ###############################

#

# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

# WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however

# in order to mark it as "mature" we need to wait for a non trivial percentage

# of users to deploy it in production.

# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

#

# Normal Redis instances can't be part of a Redis Cluster; only nodes that are

# started as cluster nodes can. In order to start a Redis instance as a

# cluster node enable the cluster support uncommenting the following:

#

 cluster-enabled yes  //开启集群功能

将redis01 复制5份

[root@localhost redis-cluster]# cp -r redis01/ redis02

[root@localhost redis-cluster]# cp -r redis01/ redis03

[root@localhost redis-cluster]# cp -r redis01/ redis04

[root@localhost redis-cluster]# cp -r redis01/ redis05

[root@localhost redis-cluster]# cp -r redis01/ redis06

修改他们的端口为 redis02:7002 redis03:7003 redis04:7004 redis05:7005 redis06:7006

5、将集群依赖的ruby脚本redis-trib.rb 移动到 /usr/local/redis-cluster 目录下便于使用

  [root@localhost src]# pwd

  /tools/redis-3.2.8/src

  [root@localhost src]# cp *.rb /usr/local/redis-cluster

6、启动6各redis实例使用脚本启动

[root@localhost redis-cluster]# vim startRedis.sh

cd redis01/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis02/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis03/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis04/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis05/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis06/bin
./redis-server ../etc/redis.conf
cd ../../

赋予该文件可执行权限：[root@localhost redis-cluster]# chmod +x startRedis.sh

7、启动脚本并查看6各redis实例是否启动成功

[root@localhost redis-cluster]# ./startRedis.sh

[root@localhost redis-cluster]# ps -ef|grep redis

root      1634     1  0 17:11 ?        00:00:00 ./redis-server 0.0.0.0:7001 [cluster]

root      1637     1  0 17:11 ?        00:00:00 ./redis-server 0.0.0.0:7003 [cluster]

root      1644     1  0 17:11 ?        00:00:00 ./redis-server 0.0.0.0:7005 [cluster]

root      1939     1  0 17:15 ?        00:00:00 ./redis-server 0.0.0.0:7002 [cluster]

root      1943     1  0 17:15 ?        00:00:00 ./redis-server 0.0.0.0:7004 [cluster]

root      1951     1  0 17:15 ?        00:00:00 ./redis-server 0.0.0.0:7006 [cluster]

root      1955  1463  0 17:15 pts/0    00:00:00 grep redis

8、创建集群

[root@localhost redis-cluster]# ./redis-trib.rb create --replicas 1 192.168.6.190:7001 192.168.6.190:7002 192.168.6.190:7003 192.168.6.190:7004 192.168.6.190:7005 192.168.6.190:7006

[root@localhost redis-cluster]# ./redis-trib.rb create --replicas 1 192.168.6.190:7001 192.168.6.190:7002 192.168.6.190:7003 192.168.6.190:7004 192.168.6.190:7005 192.168.6.190:7006

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

192.168.6.190:7001

192.168.6.190:7002

192.168.6.190:7003

Adding replica 192.168.6.190:7004 to 192.168.6.190:7001

Adding replica 192.168.6.190:7005 to 192.168.6.190:7002

Adding replica 192.168.6.190:7006 to 192.168.6.190:7003

M: 6e8c35d55d3699afe0b10317cf9c199e7df1ae50 192.168.6.190:7001

   slots:0-5460 (5461 slots) master     //slots槽的分配

M: 4cc06e1060369a16f70d4f8097c065bf886162a3 192.168.6.190:7002

   slots:5461-10922 (5462 slots) master

M: 532355022ea3473834726e7512270025b7eddae3 192.168.6.190:7003

   slots:10923-16383 (5461 slots) master

S: de789ed503705e7a5a2e5fc41adb7ed3290de08c 192.168.6.190:7004

   replicates 6e8c35d55d3699afe0b10317cf9c199e7df1ae50

S: c1261800c3a7e74ff3b2fda22b07ef994f7db4a7 192.168.6.190:7005

   replicates 4cc06e1060369a16f70d4f8097c065bf886162a3

S: 07910c0d839e4fcd6139598edd4953fcd32adb29 192.168.6.190:7006

   replicates 532355022ea3473834726e7512270025b7eddae3

Can I set the above configuration? (type 'yes' to accept): yes  输入yes创建集群

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join....

>>> Performing Cluster Check (using node 192.168.6.190:7001)

M: 6e8c35d55d3699afe0b10317cf9c199e7df1ae50 192.168.6.190:7001   M:主节点  S:从节点

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 532355022ea3473834726e7512270025b7eddae3 192.168.6.190:7003

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: de789ed503705e7a5a2e5fc41adb7ed3290de08c 192.168.6.190:7004

   slots: (0 slots) slave

   replicates 6e8c35d55d3699afe0b10317cf9c199e7df1ae50

S: c1261800c3a7e74ff3b2fda22b07ef994f7db4a7 192.168.6.190:7005

   slots: (0 slots) slave

   replicates 4cc06e1060369a16f70d4f8097c065bf886162a3

M: 4cc06e1060369a16f70d4f8097c065bf886162a3 192.168.6.190:7002

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 07910c0d839e4fcd6139598edd4953fcd32adb29 192.168.6.190:7006

   slots: (0 slots) slave

   replicates 532355022ea3473834726e7512270025b7eddae3

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

9、测试集群

使用6各redis实例中任意一个客户端链接集群：

[root@localhost redis-cluster]# redis02/bin/redis-cli -h 192.168.6.190 -p 7002 -c

[root@localhost redis-cluster]# redis02/bin/redis-cli -h 192.168.6.190 -p 7002 -c

192.168.6.190:7002> set age 100

-> Redirected to slot [741] located at 192.168.6.190:7001

OK

在这里我们登陆了集群的7002redis服务器，age的hash值是741于是就把该数据存储在了7001节点上，应为该节点的槽范围是（slots:0-5460）。

10、关闭集群

我们创建一个脚本文件redisShutdown.sh

redis01/bin/redis-cli -p 7001 shutdown

redis02/bin/redis-cli -p 7002 shutdown

redis03/bin/redis-cli -p 7003 shutdown

redis04/bin/redis-cli -p 7004 shutdown

redis05/bin/redis-cli -p 7005 shutdown

redis06/bin/redis-cli -p 7006 shutdown

到此集群搭建完毕。 ps:单机版redis默认有16个数据库，集群中只有一个数据库即0号库