rabbitmq+ keepalived+haproxy高可用集群详细命令

公司要用rabbitmq研究了两周，特把

rabbitmq 高可用的研究成果备下

后续会更新封装的类库

安装erlang

wget http://www.gelou.me/yum/erlang-18.3-1.el6.x86_64.rpm

yum install erlang-18.3-1.el6.x86_64.rpm

安装rabbitmq rpm包：

wget http://www.gelou.me/yum/rabbitmq-server-3.6.1-1.noarch.rpm

rpm --import http://www.rabbitmq.com/rabbitmq-signing-key-public.asc

yum install rabbitmq-server-3.6.1-1.noarch.rpm

启动rabbitmq，并验证启动情况

rabbitmq-server --detached &ps aux |grep rabbitm

以服务的方式启动

service rabbitmq-server start

开启系列防火墙端口

/sbin/iptables -I INPUT -p tcp --dport 5672 -j ACCEPT

/sbin/iptables -I INPUT -p tcp --dport 4369 -j ACCEPT

/sbin/iptables -I INPUT -p tcp --dport 25672 -j ACCEPT

/sbin/iptables -I INPUT -p tcp --dport 15672 -j ACCEPT

iptables -I INPUT -p tcp --dport 8888 -j ACCEPT

iptables -I INPUT -p tcp --dport 5670 -j ACCEPT

iptables -I INPUT -p tcp --dport 5670 -j ACCEPT

/etc/rc.d/init.d/iptables save

/etc/init.d/iptables restart

/etc/init.d/iptables status

设置开机启动

rpm -aq |grep chkconfig

export PATH=/sbin:$PATH

chkconfig

chkconfig rabbitmq-server on

安装web管理

rabbitmq-plugins enable rabbitmq_management

访问地址

http://172.20.16.57:15672/#/

重复安装第二台等

集群命令重复安装第二台等

vim /etc/hosts

192.168.43.74 TD-YichehuiLinu

192.168.87.5  YichehuiLinux

重启

reboot

chmod 700 /var/lib/rabbitmq/.erlang.cookie

echo -n "AZVOCZYZZBVFLBPTBXU" > /var/lib/rabbitmq/.erlang.cookie

chmod 400 /var/lib/rabbitmq/.erlang.cookie

ps -ef | grep ^rabbitmq | cut -c 9-16 | xargs kill -9

rabbitmq-server --detached &ps aux |grep rabbitm

机器全启动后分别执行

rabbitmqctl stop_app

rabbitmqctl join_cluster rabbit@各机器名

rabbitmqctl start_app

rabbitmqctl cluster_status

设置policy，以ha.开头的队列将会被镜像到集群其他所有节点,一个节点挂掉然后重启后会自动同步队列消息

必须将整个RabbitMQ集群的状态设置为镜像模式， 节点都执行

rabbitmqctl set_policy ha-all-queue "^ha\." '{"ha-mode":"all","ha-sync-mode":"automatic"}'

备用命令

rabbitmqctl stop_app

rabbitmqctl change_cluster_node_type disc

或

rabbitmqctl change_cluster_node_type ram

rabbitmqctl start_app

添加用户名admin,密码admin

rabbitmqctl add_user zhang 1234

设置admin用户tags为管理员

rabbitmqctl set_user_tags zhang administrator

设置admin用户配置、写、读的权限

rabbitmqctl set_permissions -p / zhang ".*" ".*" ".*"

haproxy搭建

1、安装haproxy

yum install haproxy

2、安装rsyslog

a. 检查rsyslog是否已安装

rpm -q rsyslog

b. 添加haproxy的log配置

cd /etc/rsyslog.d

vim haproxy.conf

=== 文件内容

$ModLoad imudp

$UDPServerRun 514

local2.* /var/log/haproxy.log

local3.* /var/log/haproxy.log

local10.* /var/log/haproxy.log

==========

c. 编辑/etc/sysconfig/rsyslog

vim /etc/sysconfig/rsyslog

添加内容：

SYSLOGD_OPTIONS="-c 2 -r -m 0"

d. 创建日志文件,并授权

cd /var/log

touch haproxy.log

chmod a+w haproxy.log

e. 重启rsyslog服务用法：

/etc/init.d/rsyslog {start|stop|restart|condrestart|try-restart|reload|force-reload|status}

/etc/init.d/rsyslog restart

配置所有机器

4、配置haproxy

> /etc/haproxy/haproxy.cfg

vim /etc/haproxy/haproxy.cfg

#---------------------------------------------------------------------

# Example configuration for a possible web application. See the

# full configuration options online.

#

# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt

#

#---------------------------------------------------------------------

#---------------------------------------------------------------------

# Global settings

#---------------------------------------------------------------------

global

# to have these messages end up in /var/log/haproxy.log you will

# need to:

#

# 1) configure syslog to accept network log events. This is done

# by adding the '-r' option to the SYSLOGD_OPTIONS in

# /etc/sysconfig/syslog

#

# 2) configure local2 events to go to the /var/log/haproxy.log

# file. A line like the following can be added to

# /etc/sysconfig/syslog

#

# local2.* /var/log/haproxy.log

#

log 127.0.0.1 local2 notice

chroot /var/lib/haproxy

pidfile /var/run/haproxy.pid

maxconn 4000

user haproxy

group haproxy

daemon

# turn on stats unix socket

stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------

# common defaults that all the 'listen' and 'backend' sections will

# use if not designated in their block

#---------------------------------------------------------------------

defaults

mode tcp

option tcplog

option dontlognull

option http-server-close

option redispatch

retries 3

timeout http-request 10s

timeout queue 1m

timeout connect 10s

timeout client 1m

timeout server 1m

timeout http-keep-alive 10s

timeout check 10s

maxconn 3000

###haproxy statistics monitor by laijingli 20160222

listen statics 0.0.0.0:8888

mode http

log 127.0.0.1 local0 debug

transparent

stats refresh 60s

stats uri / haproxy-stats

stats realm Haproxy \ statistic

stats auth zhang:1234

#---------------------------------------------------------------------

# main frontend which proxys to the backends

#---------------------------------------------------------------------

listen rabbitmq_cluster

bind 0.0.0.0:5670

mode tcp

balance roundrobin

server ych51  172.20.16.51:5672 check inter 5000 rise 2 fall 2

server ych50  172.20.16.50:5672 check inter 5000 rise 2 fall 2

server TD-YichehuiLinu  192.168.43.74:5672 check inter 5000 rise 2 fall 2

server YichehuiLinux  192.168.87.5:5672 check inter 5000 rise 2 fall 2

server YichehuiLinux  192.168.87.5:9200 check inter 5000 rise 2 fall 2

5、 启动haproxy

/etc/init.d/haproxy restart

6、 备用命令停止haproxy

killall haproxy

http://192.168.87.5:8888/

http://192.168.43.74:15672/#/

三、安装和设置keepalived

1、安装keepalived

yum install keepalived

chkconfig --add keepalived

chkconfig --level 35 keepalived on

2、修改配置文件

配置keepalived

>  /etc/keepalived/keepalived.conf

vim  /etc/keepalived/keepalived.conf

内容：

=====================================================================

! Configuration File for keepalived

# 全局配置，demo中配置了邮件信息，此处因为调试中，暂时没有添加

global_defs {

}

# 集群资源监控，组合track_script进行

vrrp_script check_haproxy {

script "killall -0 haproxy"

interval 2

}

vrrp_instance HAPROXY_HA {

# 设置当前主机为主节点，如果是备用节点，则设置为BACKUP

# 备用节点时，设置为：

# state BACKUP

state MASTER

# 指定HA监测网络接口，可以用ifconfig查看来决定设置哪一个

interface eth1

# 虚拟路由标识，同一个VRRP实例要使用同一个标识，主备机

virtual_router_id 80

# 因为当前环境中VRRP组播有问题，改为使用单播发送VRRP报文

# 这个地方需要关注，之前未做此设置，结果主备节点互相不能发现，因此主备节点都升级成了MASTER，并且绑定了VIP

# 备用节点时，内容为：

# unicast_src_ip 192.168.200.199

# unicast_peer {

# 192.168.200.200

# }

# 设置优先级，确保主节点的优先级高过备用节点

# 备用节点时，设置为：

priority 80

# priority 100

# 用于设定主备节点间同步检查时间间隔

advert_int 2

# 设置高可用集群中不抢占功能，在主机down后，从机接管，当主机重新恢复后，设置此功能，备机将继续提供服务，从而避免因切换导致的隐患

nopreempt

# 设置主备节点间的通信验证类型及密码，同一个VRRP实例中需一致

authentication {

auth_type PASS

auth_pass 1234

}

# 当keepalived切换状态到MASTER时，执行脚本

notify_master "/etc/keepalived/master.sh"

# 当keepalived切换状态到BACKUP时，执行脚本

notify_backup "/etc/keepalived/backup.sh"

# 当keepalived切换状态到FAULT时，执行脚本

notify_fault "/etc/keepalived/fault.sh"

# 当keepalived切换状态到STOP时，执行脚本

notify_fault "/etc/keepalived/stop.sh"

# 集群资源监控，组合vrrp_script进行

track_script {

check_haproxy

}

# 设置虚拟IP地址，当keepalived状态切换为MASTER时，此IP会自动添加到系统中

# 当状态切换到BACKUP时，此IP会自动从系统中删除

# 可以通过命令ip add查看切换后的状态

virtual_ipaddress {

192.168.43.201

}

}

====================================================================

配置全部

3、启动服务

因为是为了实现haproxy的高可用，启动时需要顺序启动：

(1) 启动haproxy：

haproxy -f /etc/haproxy/haproxy.cfg

(2) 启动keeepalived：先启动master节点，后启动BACKUP节点

/etc/init.d/keepalived restart

4、停止keeepalived服务

/etc/init.d/keepalived stop

大哥广告阁楼