MySQL8.0使用mysqlsh配置主从复制 InnoDB ReplicaSet

InnoDB ReplicaSet

InnoDB ReplicaSet 由一个主节点和多个从节点构成. 可以使用ReplicaSet对象和AdminAPI操作管理复制集, 例如检查InnoDB复制集的状态, 并在发生故障时手动故障转移到新的主服务器.

与InnoDB cluster类似, MySQL Router 支持针对InnoDB ReplicaSet 的引导, 这意味着可以自动配置MySQL Router以使用 InnoDB ReplicaSet, 而无需手动配置文件. 这使得InnoDB ReplicaSet 成为一种快速简便的方法, 可以启动和运行 MySQL 复制和 MySQL Router, 非常适合扩展读取, 并在不需要InnoDB集群提供高可用性的用例中提供手动故障转移功能.

InnoDB ReplicaSet限制

与InnoDB集群相比, InnoDB ReplicaSet 有几个限制, 包括:

1. 没有故障自动切换功能. 在主库不可用的情况下, 需要使用AdminApi手动触发故障转移
2. 无法防止由于意外或者不可用而导致的数据丢失, 发生故障时候没有应用的事务可能有丢失现象
3. 无法防止意外退出之后的数据不一致现象；在主节点因为网络问题而短暂失联时候, 如果某个从节点提升为主节点, 则可能发生脑裂现象
4. Innodb 副本集不支持多主模式
5. Innodb副本集基于异步复制, 不能像 MGR 那样进行流控, 因此读的扩展性会一定程度上受限
6. 所有 Secondary 都要从 Primary复制, 因此可能会对 Primary 的负载产生一定的影响

部署

和部署 InnoDB cluster的方式相似, 首先配置一些MySQL服务器实例, 运行 InnoDB ReplicaSet 的前提条件

1. 仅支持MySQL8.0及以上版本；
2. 仅支持GTID的复制方法
3. 仅支持row格式的binlog，不支持statement格式的binlog
4. 不支持复制过滤器
5. 不允许建立额外的复制通道
6. 副本集primary节点只有一个，secondary节点可以有多个，MySQL Router会对每个节点进行监控
7. 副本集必须完全由MySQL Shell管理，不支持在MySQL Shell之外对实例进行配置和更改

下载和安装mysql-shell

访问 https://dev.mysql.com/downloads/shell/ 下载 Linux Generice 版本, 因为这个版本自带支持此版本的Python3. 如果下载rpm包, 在Centos8下安装时会告诉你缺libpython3.9, 可是Centos8自带的版本才到3.6, 自己编译的话还要再装一堆东西, 不如直接用Generic版本.

解压并放到/opt/mysql-shell下

tar xvf mysql-shell-8.0.26-linux-glibc2.12-x86-64bit.tar.gz
cd /opt/
mkdir mysql-shell
cd mysql-shell
mv ~/backup/mysql-shell-8.0.26-linux-glibc2.12-x86-64bit .
ln -s mysql-shell-8.0.26-linux-glibc2.12-x86-64bit latest

也可以软链到 /usr/bin 下, 这样就可以直接用命令行运行了

cd /usr/bin
ln -s /opt/mysql-shell/latest/bin/mysqlsh mysqlsh

部署

运行mysqlsh

$ /opt/mysql-shell/latest/bin/mysqlsh
MySQL Shell 8.0.26

Copyright (c) 2016, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates.
Other names may be trademarks of their respective owners.

Type '\help' or '\?' for help; '\quit' to exit.
 MySQL  JS >

设置第一个节点

 MySQL  192.168.255.4:33060+ ssl  JS > dba.configureReplicaSetInstance('',{clusterAdmin:"'repladmin'@'192.168.255.0/24'"});
Dba.configureReplicaSetInstance: Argument #1: Invalid URI: empty. (ArgumentError)
 MySQL  192.168.255.4:33060+ ssl  JS > dba.configureReplicaSetInstance('admin@192.168.255.4:3306',{clusterAdmin:"'repladmin'@'192.168.255.0/24'"});
Please provide the password for 'admin@192.168.255.4:3306': ************
Save password for 'admin@192.168.255.4:3306'? [Y]es/[N]o/Ne[v]er (default No):
Configuring local MySQL instance listening at port 3306 for use in an InnoDB ReplicaSet...

This instance reports its own address as db01:3306
Clients and other cluster members will communicate with it through this address by default. If this is not correct, the report_host MySQL system variable should be changed.
Password for new account: *********
Confirm password: *********

applierWorkerThreads will be set to the default value of 4.

NOTE: Some configuration options need to be fixed:
+----------------------------------------+---------------+----------------+--------------------------------------------------+
| Variable                               | Current Value | Required Value | Note                                             |
+----------------------------------------+---------------+----------------+--------------------------------------------------+
| binlog_transaction_dependency_tracking | COMMIT_ORDER  | WRITESET       | Update the server variable                       |
| enforce_gtid_consistency               | OFF           | ON             | Update read-only variable and restart the server |
| gtid_mode                              | OFF           | ON             | Update read-only variable and restart the server |
| replica_parallel_type                  | DATABASE      | LOGICAL_CLOCK  | Update the server variable                       |
| replica_preserve_commit_order          | OFF           | ON             | Update the server variable                       |
| server_id                              | 1             | <unique ID>    | Update read-only variable and restart the server |
+----------------------------------------+---------------+----------------+--------------------------------------------------+

Some variables need to be changed, but cannot be done dynamically on the server.
Do you want to perform the required configuration changes? [y/n]: y
Do you want to restart the instance after configuring it? [y/n]: y
Cluster admin user 'repladmin'@'192.168.255.0/24' created.
Configuring instance...
The instance 'db01:3306' was configured to be used in an InnoDB ReplicaSet.
Restarting MySQL...
NOTE: MySQL server at db01:3306 was restarted.

创建ReplicaSet

-- 第一次执行未成功, 待恢复连接
 MySQL  192.168.255.4:33060+ ssl  JS > var rs = dba.createReplicaSet("mytest_db")
Dba.createReplicaSet: MySQL server has gone away (MYSQLSH 2006)
The global session got disconnected..
Attempting to reconnect to 'mysqlx://admin@192.168.255.4:33060'..
The global session was successfully reconnected.
 MySQL  192.168.255.4:33060+ ssl  JS > rs.status()
TypeError: Cannot read property 'status' of undefined
-- 第二次才成功
 MySQL  192.168.255.4:33060+ ssl  JS > var rs = dba.createReplicaSet("mytest_db")
A new replicaset with instance 'db01:3306' will be created.

* Checking MySQL instance at db01:3306

This instance reports its own address as db01:3306
db01:3306: Instance configuration is suitable.

* Updating metadata...

ReplicaSet object successfully created for db01:3306.
Use rs.addInstance() to add more asynchronously replicated instances to this replicaset and rs.status() to check its status.
-- 查看rs状态
 MySQL  192.168.255.4:33060+ ssl  JS > rs.status()
{
    "replicaSet": {
        "name": "mytest_db",
        "primary": "db01:3306",
        "status": "AVAILABLE",
        "statusText": "All instances available.",
        "topology": {
            "db01:3306": {
                "address": "db01:3306",
                "instanceRole": "PRIMARY",
                "mode": "R/W",
                "status": "ONLINE"
            }
        },
        "type": "ASYNC"
    }
}

初始化另一个节点

Dba.configureReplicaSetInstance: Access denied for user 'admin'@'192.168.255.4' (using password: YES) (MySQL Error 1045)
 MySQL  192.168.255.4:33060+ ssl  JS > dba.configureReplicaSetInstance('admin@192.168.255.5:3306',{clusterAdmin:"'repladmin'@'192.168.255.0/24'"});
Please provide the password for 'admin@192.168.255.5:3306': ************
Save password for 'admin@192.168.255.5:3306'? [Y]es/[N]o/Ne[v]er (default No):
Configuring MySQL instance at db02:3306 for use in an InnoDB ReplicaSet...

This instance reports its own address as db02:3306
Clients and other cluster members will communicate with it through this address by default. If this is not correct, the report_host MySQL system variable should be changed.
Password for new account: *********
Confirm password: *********

applierWorkerThreads will be set to the default value of 4.

NOTE: Some configuration options need to be fixed:
+----------------------------------------+---------------+----------------+--------------------------------------------------+
| Variable                               | Current Value | Required Value | Note                                             |
+----------------------------------------+---------------+----------------+--------------------------------------------------+
| binlog_transaction_dependency_tracking | COMMIT_ORDER  | WRITESET       | Update the server variable                       |
| enforce_gtid_consistency               | OFF           | ON             | Update read-only variable and restart the server |
| gtid_mode                              | OFF           | ON             | Update read-only variable and restart the server |
| replica_parallel_type                  | DATABASE      | LOGICAL_CLOCK  | Update the server variable                       |
| replica_preserve_commit_order          | OFF           | ON             | Update the server variable                       |
| server_id                              | 1             | <unique ID>    | Update read-only variable and restart the server |
+----------------------------------------+---------------+----------------+--------------------------------------------------+

Some variables need to be changed, but cannot be done dynamically on the server.
Do you want to perform the required configuration changes? [y/n]: y
Do you want to restart the instance after configuring it? [y/n]: y
Cluster admin user 'repladmin'@'192.168.255.0/24' created.
Configuring instance...
The instance 'db02:3306' was configured to be used in an InnoDB ReplicaSet.
Restarting MySQL...
NOTE: MySQL server at db02:3306 was restarted.

将另一个节点加入

注意, 两个机器的hostname和IP映射关系必须出现在对方的/etc/hosts里, 否则中间会出错, 因为互相默认是使用hostname来连接的.

 MySQL  192.168.255.4:33060+ ssl  JS > rs.addInstance("192.168.255.5:3306")
Adding instance to the replicaset...

* Performing validation checks

This instance reports its own address as db02:3306
db02:3306: Instance configuration is suitable.

* Checking async replication topology...

* Checking transaction state of the instance...

NOTE: The target instance 'db02:3306' has not been pre-provisioned (GTID set is empty). The Shell is unable to decide whether replication can completely recover its state.
The safest and most convenient way to provision a new instance is through automatic clone provisioning, which will completely overwrite the state of 'db02:3306' with a physical snapshot from an existing replicaset member. To use this method by default, set the 'recoveryMethod' option to 'clone'.

WARNING: It should be safe to rely on replication to incrementally recover the state of the new instance if you are sure all updates ever executed in the replicaset were done with GTIDs enabled, there are no purged transactions and the new instance contains the same GTID set as the replicaset or a subset of it. To use this method by default, set the 'recoveryMethod' option to 'incremental'.

Please select a recovery method [C]lone/[I]ncremental recovery/[A]bort (default Clone): C
* Updating topology
Waiting for clone process of the new member to complete. Press ^C to abort the operation.
* Waiting for clone to finish...
NOTE: db02:3306 is being cloned from db01:3306
** Stage DROP DATA: Completed
** Clone Transfer
    FILE COPY  ####################################################  100%  Compl    PAGE COPY  ####################################################  100%  Compl    REDO COPY  ####################################################  100%  Completed
NOTE: db02:3306 is shutting down...

* Waiting for server restart... ready
* db02:3306 has restarted, waiting for clone to finish...
** Stage RESTART: Completed
* Clone process has finished: 75.35 MB transferred in about 1 second (~75.35 MB/s)

** Configuring db02:3306 to replicate from db01:3306
** Waiting for new instance to synchronize with PRIMARY...

The instance 'db02:3306' was added to the replicaset and is replicating from db01:3306.

检查状态

 MySQL  192.168.255.4:33060+ ssl  JS > rs.status()
{
    "replicaSet": {
        "name": "mytest_db",
        "primary": "db01:3306",
        "status": "AVAILABLE",
        "statusText": "All instances available.",
        "topology": {
            "db01:3306": {
                "address": "db01:3306",
                "instanceRole": "PRIMARY",
                "mode": "R/W",
                "status": "ONLINE"
            },
            "db02:3306": {
                "address": "db02:3306",
                "instanceRole": "SECONDARY",
                "mode": "R/O",
                "replication": {
                    "applierStatus": "APPLIED_ALL",
                    "applierThreadState": "Waiting for an event from Coordinator",
                    "applierWorkerThreads": 4,
                    "receiverStatus": "ON",
                    "receiverThreadState": "Waiting for source to send event",
                    "replicationLag": null
                },
                "status": "ONLINE"
            }
        },
        "type": "ASYNC"
    }
}

第二次运行

第二次要先连db, 然后执行var rs = dba.getReplicaSet()读取ReplicaSet

[root@db01 ~]# mysqlsh
MySQL Shell 8.0.26

Copyright (c) 2016, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates.
Other names may be trademarks of their respective owners.

Type '\help' or '\?' for help; '\quit' to exit.
 MySQL  JS > var rs = dba.get
getCluster()     getReplicaSet()
 MySQL  JS > var rs = dba.getReplicaSet()
Dba.getReplicaSet: An open session is required to perform this operation. (RuntimeError)
 MySQL  JS > \connect admin@192.168.255.4
Creating a session to 'admin@192.168.255.4'
Please provide the password for 'admin@192.168.255.4': ************
Save password for 'admin@192.168.255.4'? [Y]es/[N]o/Ne[v]er (default No):
Fetching schema names for autocompletion... Press ^C to stop.
Your MySQL connection id is 41 (X protocol)
Server version: 8.0.26 MySQL Community Server - GPL
No default schema selected; type \use <schema> to set one.
 MySQL  192.168.255.4:33060+ ssl  JS > var rs = dba.getReplicaSet()
You are connected to a member of replicaset 'mytest_db'.
 MySQL  192.168.255.4:33060+ ssl  JS > rs.status()
{
    "replicaSet": {
        "name": "mytest_db",
        "primary": "db01:3306",
        "status": "AVAILABLE",
        "statusText": "All instances available.",
        "topology": {
            "db01:3306": {
                "address": "db01:3306",
                "instanceRole": "PRIMARY",
                "mode": "R/W",
                "status": "ONLINE"
            },
            "db02:3306": {
                "address": "db02:3306",
                "instanceRole": "SECONDARY",
                "mode": "R/O",
                "replication": {
                    "applierStatus": "APPLIED_ALL",
                    "applierThreadState": "Waiting for an event from Coordinator",
                    "applierWorkerThreads": 4,
                    "receiverStatus": "ON",
                    "receiverThreadState": "Waiting for source to send event",
                    "replicationLag": null
                },
                "status": "ONLINE"
            }
        },
        "type": "ASYNC"
    }
}

参考

• MySQL原生高可用方案之InnoDB ReplicaSet https://cloud.tencent.com/developer/article/1816453
  • https://cloud.tencent.com/developer/article/1816455
  • https://cloud.tencent.com/developer/article/1816462
• MySQL8.0.19 InnoDB ReplicaSet介绍与搭建 https://cloud.tencent.com/developer/article/1578567
• MySQL8.0.19 InnoDB ReplicaSet https://zhuanlan.zhihu.com/p/106701112
• GTID介绍, 很长 https://www.cnblogs.com/kevingrace/p/5569753.html