Keepalived与MySQL互为主从自动切换配置

为解决Mysql数据库单点问题，实现两台MySQL数据库互为主备，双向replication。当一Master出现问题，则将Slave切换为Master继续工作.

环境说明

系统版本：CentOS Linux release 7.6.1810 (Core)

MySQL版本：mysql  Ver 14.14 Distrib 5.7.27

keepalived版本：Keepalived v1.2.13

序号 　　  服务器IP 　　　　用途

1 　　　　192.168.158.10    Master
2 　　　　192.168.158.20    Slave
3 　　　　192.168.158.30    VIP

一、MySQL互为主从配置

1.> 两台安装相同版本的MySQL数据库.
2.> 主备机NTP时钟同步
3.> 双机互信配置ssh免密认证
4.> 数据库配置(Master的配置和Slave的配置server-id不能一致,别的都可以一样)
4.1> 修改Master主机上MySQL数据库的配置文件，然后新启动MySQL

#vim /ect/my.cnf
[mysqld]
log-bin=mysql-bin
server-id=
expire_logs_days =
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
validate_password=off         #关闭密码安全策略
default_password_lifetime=     #设置密码不过期
log_bin=/var/log/mysql/mysql-bin

4.2> 修改Slave主机上MySQL数据库的配置文件，然后新启动MySQL

#vim /ect/my.cnf

[mysqld]
log-bin=mysql-bin
server-id=
expire_logs_days =
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
validate_password=off
default_password_lifetime=
log_bin=/var/log/mysql/mysql-bin

5.> 启动MySQL服务

# systemctl start mysqld

6.> 查询相关状态，以Master主机为例，如下

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin. |       |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
 row in set (0.00 sec)

7.> 创建复制账号并同步
7.1> 在Master库和Slave库分别执行，创建数据同步复制账号.

mysql> GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO replication@'%' IDENTIFIED BY 'replication';
mysql> flush  privileges;

7.2> 7.2> 在Master主机上,执行同步操作(注意master_host参数主备机相互指向),如下：

mysql> change master to master_host='192.168.158.20',master_port=,master_user='replication',master_password='replication',master_log_file='mysql-bin.000001',master_log_pos=;
mysql> start slave;

7.3> 在Slave主机上,执行同步操作(注意master_host参数主备机相互指向),如下：

mysql> change master to master_host='192.168.158.10',master_port=,master_user='replication',master_password='replication',master_log_file='mysql-bin.000001',master_log_pos=;
mysql> start slave;

7.4> 在Master、Slave主机上，查询同步状态“show slave status\G”，检查结果中Slave_IO_Running: Yes和Slave_SQL_Running: Yes，否则有异常。

8.> 配置密文命令访问(两台主机都配置)
Mysql数据库使用mysql或mysqldump等相关命令时，需要在命令行界面输入密码，当使用脚本时，在脚本里填写密码显然不太安全，因此可以设置Mysql的密文文件。

# mysql_config_editor set --login-path=local --user=root --port= --password
# mysql_config_editor print --all

9.> 创建切换脚本

切换脚本规划，如本次是mysql切换，因此在该目录下创建mysql目录,将所有切换脚本放在/home/mysql目录下,本次相关脚本说明如下：

进入/home/mysql目录，如下文件：

Logs   　　       //存储日志的文件目录

mybackup.sh   //清空slave配置，重新获取远程日志文件及Pos,并开启同步

mycheck.sh     //检查mysql运行状态，如果运行正常，退出。如果运行不正常调用pkill keepalived

mymaster.sh    //先判断同步复制是否执行完成，如果未执行完成等待1分钟后，停止同步（stop slave;），并且记录切换后的日志和pos

.mysqlenv       //脚本运行环境文件

mystop.sh       //设置参数保证数据不丢失，最后检查看是否还有写操作，最后1分钟退出

syncposfile     //每次切换后，Master最后一次File值和Position值。

10.环境文件

10.1> Master主机端的环境文件

[root@localhost mysql]# vim .mysqlenv

MYSQL=/usr/bin/mysql
MYSQL_CMD="--login-path=local"
#远端主机的IP地址
REMOTE_IP=192.168.158.20
export mysql="$MYSQL $MYSQL_CMD "

10.2> Slave主机端的环境文件

[root@localhost mysql]# vim .mysqlenv
MYSQL=/usr/bin/mysql
MYSQL_CMD="--login-path=local"
#远端主机的IP地址
REMOTE_IP=192.168.158.10
export mysql="$MYSQL $MYSQL_CMD"

11.> 服务检查脚本

11.1> mycheck.sh

[root@localhost mysql]# vim mycheck.sh
#!/bin/sh

##################################################
#File Name  : mycheck.sh
#Description: mysql is working MYSQL_OK is
#             mysql is down MYSQL_OK is
##################################################

BASEPATH=/home/mysql
LOGSPATH=$BASEPATH/logs
source $BASEPATH/.mysqlenv

CHECK_TIME=
MYSQL_OK=
##################################################################
function check_mysql_helth (){
  $mysql -e "show status;" >/dev/null >&
  if [ $? ==  ]
  then
    MYSQL_OK=
  else
    MYSQL_OK=
    #systemctl status keepalived
 fi
 return $MYSQL_OK
}

#check_mysql_helth
while [ $CHECK_TIME -ne  ] #不等于
do
    let "CHECK_TIME -= 1"
    check_mysql_helth
    if [ $MYSQL_OK =  ] ; then
    CHECK_TIME=
    echo "$(date "+%Y-%m-%d %H:%M:%S") The scripts mycheck.sh is running ..." >> $LOGSPATH/mysql_switch.log
    exit
    fi
    if [ $MYSQL_OK -eq  ] && [ $CHECK_TIME -eq  ] #等于
    then
    systemctl stop keepalived
    echo "$(date "+%Y-%m-%d %H:%M:%S") The mycheck.sh, mysql is down, after switch..." >> $LOGSPATH/mysql_switch.log
    exit
    fi
    sleep 　　
done
[root@localhost mysql]# 

11.2> 切换脚本

[root@localhost mysql]# vim mymaster.sh
#!/bin/sh

##################################################
#File Name  : mymaster.sh
#Description: First determine whether synchronous
#             replication is performed, and if no
#             execution is completed, wait for
#             minutes. Log logs and POS after
#             switching, and record files synchronously.
##################################################

BASEPATH=/home/mysql
LOGSPATH=$BASEPATH/logs
source $BASEPATH/.mysqlenv

$mysql -e "show slave status\G" > $LOGSPATH/mysqlslave.states
Master_Log_File=`cat $LOGSPATH/mysqlslave.states | grep -w Master_Log_File | awk -F": " '{print $2}'`
Relay_Master_Log_File=`cat $LOGSPATH/mysqlslave.states | grep -w Relay_Master_Log_File | awk -F": " '{print $2}'`
Read_Master_Log_Pos=`cat $LOGSPATH/mysqlslave.states | grep -w Read_Master_Log_Pos | awk -F": " '{print $2}'`
Exec_Master_Log_Pos=`cat $LOGSPATH/mysqlslave.states | grep -w Exec_Master_Log_Pos | awk -F": " '{print $2}'`
i=

while true
do
    if [ $Master_Log_File = $Relay_Master_Log_File ] && [ $Read_Master_Log_Pos -eq $Exec_Master_Log_Pos ];then
        echo "$(date "+%Y-%m-%d %H:%M:%S") The mymaster.sh, slave sync ok... " >> $LOGSPATH/mysql_switch.log
        break
    else
        sleep
        if [ $i -gt  ];then
            break
        fi
        continue
        let i++
    fi
done

$mysql -e "stop slave;"
$mysql -e "set global innodb_support_xa=0;"
$mysql -e "set global sync_binlog=0;"
$mysql -e "set global innodb_flush_log_at_trx_commit=0;"
$mysql -e "flush logs;GRANT ALL PRIVILEGES ON *.* TO 'replication'@'%' IDENTIFIED BY 'replication';flush privileges;"
$mysql -e "show master status;" > $LOGSPATH/master_status_$(date "+%y%m%d-%H%M").txt

# sync pos file
/usr/bin/scp $LOGSPATH/master_status_$(date "+%y%m%d-%H%M").txt root@$REMOTE_IP:$BASEPATH/syncposfile/backup_master.status
echo "$(date "+%Y-%m-%d %H:%M:%S") The mymaster.sh, Sync pos file sucess." >> $LOGSPATH/mysql_switch.log
[root@localhost mysql]#

11.3> 回切脚本

[root@localhost mysql]# vim mybackup.sh
#!/bin/sh

##################################################
#File Name  : mybackup.sh
#Description: Empty the slave configuration, retrieve
#             the remote log file and Pos, and open
#             the synchronization
##################################################

BASEPATH=/home/mysql
LOGSPATH=$BASEPATH/logs
source $BASEPATH/.mysqlenv

$mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'replication'@'%' IDENTIFIED BY 'replication';flush privileges;"
$mysql -e "set global innodb_support_xa=0;"
$mysql -e "set global sync_binlog=0;"
$mysql -e "set global innodb_flush_log_at_trx_commit=0;"
$mysql -e "flush logs;"
$mysql -e "reset slave all;"

if [ -f $BASEPATH/syncposfile/backup_master.status ];then
        New_ReM_File=`cat $BASEPATH/syncposfile/backup_master.status | grep -v File |awk '{print $1}'`
        New_ReM_Position=`cat $BASEPATH/syncposfile/backup_master.status | grep -v File |awk '{print $2}'`
        echo "$(date "+%Y-%m-%d %H:%M:%S") This mybackup.sh, New_ReM_File:$New_ReM_File,New_ReM_Position:$New_ReM_Position" >> $LOGSPATH/mysql_switch.log
        $mysql -e "change master to master_host='$REMOTE_IP',master_port=3306,master_user='replication',master_password='replication',master_log_file='$New_ReM_File',master_log_pos=$New_ReM_Position;"
        $mysql -e "start slave;"
        $mysql -e "show slave status\G;" > $LOGSPATH/slave_status_$(date "+%y%m%d-%H%M").txt
        cat $LOGSPATH/slave_status_$(date "+%y%m%d-%H%M").txt >> $LOGSPATH/mysql_switch.log
        rm -f $BASEPATH/syncposfile/backup_master.status
else
    echo "$(date "+%Y-%m-%d %H:%M:%S") The scripts mybackup.sh running error..." >> $LOGSPATH/mysql_switch.log
fi
[root@localhost mysql]# 

11.4> 停止脚本

[root@localhost mysql]# vim mystop.sh
#!/bin/sh

##################################################
#File Name  : mystop.sh
#Description: Set parameters to ensure that the data
#             is not lost, and finally check to see
#             if there are still write operations,
#             the last  minutes to exit

##################################################

BASEPATH=/home/mysql
LOGSPATH=$BASEPATH/logs
source $BASEPATH/.mysqlenv

$mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'replication'@'%' IDENTIFIED BY 'replication';flush privileges;"
$mysql -e "set global innodb_support_xa=1;"
$mysql -e "set global sync_binlog=1;"
$mysql -e "set global innodb_flush_log_at_trx_commit=1;"
$mysql -e "show master status\G" > $LOGSPATH/mysqlmaster0.states
M_File1=`cat $LOGSPATH/mysqlmaster0.states | awk -F': ' '/File/{print $2}'`
M_Position1=`cat $LOGSPATH/mysqlmaster0.states | awk -F': ' '/Position/{print $2}'`
sleep
$mysql -e "show master status\G" > $LOGSPATH/mysqlmaster1.states
M_File2=`cat $LOGSPATH/mysqlmaster1.states | awk -F': ' '/File/{print $2}'`
M_Position2=`cat $LOGSPATH/mysqlmaster1.states | awk -F': ' '/Position/{print $2}'`

i=

while true
do
    if [ $M_File1 = $M_File2 ] && [ $M_Position1 -eq $M_Position2 ];then
        echo "$(date "+%Y-%m-%d %H:%M:%S") The mystop.sh, master sync ok.." >> $LOGSPATH/mysql_switch.log
        exit
    else
        sleep
        if [$i -gt  ];then
            break
        fi
        continue
        let i++
    fi
done
echo "$(date "+%Y-%m-%d %H:%M:%S") The mystop.sh, master sync exceed one minutes..." >> $LOGSPATH/mysql_switch.log
[root@localhost mysql]#

二、Keepalived安装与配置

1.两台都安装Keepalived(略)

2.切换原理

Keepalived可实现将虚拟IP地址在实体物理机上来回漂移。Keepalived在转换状态时会依照状态来呼叫配置文件中内置的定义。
当进入Master状态时会呼叫notify_master定义的脚本
当进入Backup状态时会呼叫notify_backup定义的脚本
当keepalived程序终止时呼叫notify_stop定义的脚本
当发现异常情况时进入Fault状态呼叫notify_fault定义的脚本

切换的过程如下：

1.>在Master主机上keepalived运行时执行mycheck.sh脚本不停的检查mysql的运行状态，当发现mysql停止后将keepalived进程杀掉。
2.>此时Slave主机上会接管虚拟IP地址，并调用notify_master定义的脚本
3.>当原Master主机上的mysql和keepalived进程恢复正常后，会调用notify_backup定义的脚本，此时数据库的主端还在Savle主机上。
4.>回切，关闭Slave端的keepavlied进程，会调用notify_stop脚本，同时Master主机上会调用notify_master定义的脚本。此时数据库的主端在Master主机上
5.>启动Slave端的keepavlied进程，会调用notify_backup脚本，此时完成数据同步。

3.Keepalived的配置
在Master端和Savle端均安装好keepalived后，进行配置，修改/etc/keepalived/keepalived.conf文件.
3.1> Master端配置

[root@localhost keepalived]# cat keepalived.conf
global_defs {
   router_id MySQL-HA
} 

vrrp_script check_run {
script "/home/mysql/mycheck.sh"
interval
}

vrrp_sync_group VG1 {
group {
VI_1
}
}

vrrp_instance VI_1 {
    state MASTER
    #state BACKUP
    interface enp0s3
    virtual_router_id
    priority
    advert_int
    #nopreempt
    authentication {
        auth_type PASS
        auth_pass
    }
    track_script {
    check_run
    }
    notify_master /home/mysql/mymaster.sh
    notify_backup /home/mysql/mybackup.sh
    notify_stop /home/mysql/mystop.sh

    virtual_ipaddress {
        192.168.158.30/
    }

  }

3.2> Slave端配置

[root@localhost keepalived]# cat keepalived.conf
global_defs {
   router_id MySQL-HA
} 

vrrp_script check_run {
script "/home/mysql/mycheck.sh"
interval
}

vrrp_sync_group VG1 {
group {
VI_1
}
}

vrrp_instance VI_1 {
    state MASTER
    #state BACKUP
    interface enp0s3
    virtual_router_id
    priority
    advert_int
    #nopreempt
    authentication {
        auth_type PASS
        auth_pass
    }
    track_script {
    check_run
    }
    notify_master /home/mysql/mymaster.sh
    notify_backup /home/mysql/mybackup.sh
    notify_stop /home/mysql/mystop.sh

    virtual_ipaddress {
        192.168.158.30/
    }
  }
[root@localhost keepalived]#

3.3> 重新启动相关服务

# systemctl restart keepalived

三、切换验证

1. 保证两台主机上面keepalived、MySQL服务都是正常启动着的.
2. 停止主端
2.1> 将MySQL进程杀死

[root@localhost ~]# systemctl stop  mysqld

2.2> 检查状态
主端查看脚本切换日志

[root@localhost ~]# tail -100f /home/mysql/logs/mysql_switch.log
......
-- :: The scripts mycheck.sh is running ...
-- :: The scripts mycheck.sh is running ...
-- :: The scripts mycheck.sh is running ...
-- :: The scripts mycheck.sh is running ...
-- :: The scripts mycheck.sh is running ...
-- :: The mycheck.sh, mysql is down, after switch...

2.3> 主端查看浮动IP地址的切换过程。

#浮动IP地址原先在Master端，如下：
# 切换后，在从Master端验查看，浮动IP已被切走到备机
# 在Slave端查看验证，确认
# 外部ping浮动IP地址效果，有一个丢包

2.4> 主端Keepalived日志/var/log/messages如下：

Aug  :: localhost systemd: Stopping MySQL Server...
Aug  :: localhost systemd: Stopped MySQL Server.
Aug  :: localhost systemd: Stopping SYSV: Start and stop Keepalived...
Aug  :: localhost Keepalived[]: Stopping Keepalived v1.2.13 (/,)
Aug  :: localhost Keepalived_vrrp[]: VRRP_Instance(VI_1) sending  priority
Aug  :: localhost Keepalived_vrrp[]: VRRP_Instance(VI_1) removing protocol VIPs.
Aug  :: localhost Keepalived_healthcheckers[]: Netlink reflector reports IP 192.168.158.30 removed
Aug  :: localhost keepalived: Stopping keepalived: [  OK  ]
Aug  :: localhost systemd: Stopped SYSV: Start and stop Keepalived.
Aug  :: localhost systemd: Started Session  of user root.
Aug  :: localhost systemd-logind: New session  of user root.
Aug  :: localhost systemd-logind: Removed session .

2.5> 备端查看切换日志/home/mysql/logs/mysql_switch.log

-- :: The scripts mycheck.sh is running ...
-- :: The mymaster.sh, slave sync ok...
-- :: The mymaster.sh, Sync pos file sucess.
-- :: The scripts mycheck.sh is running ...

2.6> 备端查看/var/log/messages.log日志

Aug  :: localhost Keepalived_vrrp[]: VRRP_Instance(VI_1) Transition to MASTER STATE
Aug  :: localhost Keepalived_vrrp[]: VRRP_Group(VG1) Syncing instances to MASTER state
Aug  :: localhost Keepalived_vrrp[]: VRRP_Instance(VI_1) Entering MASTER STATE
Aug  :: localhost Keepalived_vrrp[]: VRRP_Instance(VI_1) setting protocol VIPs.
Aug  :: localhost Keepalived_healthcheckers[]: Netlink reflector reports IP 192.168.158.30 added
Aug  :: localhost Keepalived_vrrp[]: VRRP_Instance(VI_1) Sending gratuitous ARPs on enp0s3 for 192.168.158.30
Aug  :: localhost Keepalived_vrrp[]: VRRP_Instance(VI_1) Sending gratuitous ARPs on enp0s3 for 192.168.158.30

# mysql_config_editor set --login-path=local --user=root --port=3306 --password# mysql_config_editor print --all