gtid_executed和gtid_purged变量是如何初始化的
一、官方释义
1.1、gtid_executed、gtid_purged
https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html#sysvar_gtid_executed
• gtid_executed
When used with global scope, this variable contains a representation of the set of all transactions executed on the server and GTIDs that have been set by a SET gtid_purged statement. This is the same as the value of the Executed_Gtid_Set column in the output of SHOW MASTER STATUS and SHOW SLAVE STATUS.
• gtid_purged
The set of all transactions that have been purged from the binary log. This is a subset of the set of transactions in gtid_executed.
gtid_executed(global)：MySQL数据库已经执行过的Gtid事务，处于内存中。show master status/show slave status中的Executed_Gtid_Set也取自这里
gtid_purged(global)：由于binlog文件的删除(如purge binary logs或者超过expire_logs_days设置)已经丢失的Gtid事务，它是gtid_executed的子集
1.2、binlog_gtid_simple_recovery
https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html#sysvar_binlog_gtid_simple_recovery
binlog_gtid_simple_recovery=FALSE
• To initialize gtid_executed, binary log files are iterated from the newest file, stopping at the first binary log that has any Previous_gtids_log_event. All GTIDs from Previous_gtids_log_event and Gtid_log_events are read from this binary log file. This GTID set is stored internally and called gtids_in_binlog. The value of gtid_executed is computed as the union of this set and the GTIDs stored in the mysql.gtid_executed table.
This process could take a long time if you had a large number of binary log files without GTID events, for example created when gtid_mode=OFF.
• To initialize gtid_purged, binary log files are iterated from the oldest to the newest, stopping at the first binary log that contains either a Previous_gtids_log_event that is non-empty (that has at least one GTID) or that has at least one Gtid_log_event. From this binary log it reads Previous_gtids_log_event. This GTID set is subtracted from gtids_in_binlog and the result stored in the internal variable gtids_in_binlog_not_purged. The value of gtid_purged is initialized to the value of gtid_executed, minus gtids_in_binlog_not_purged.
binlog_gtid_simple_recovery=TRUE
which is the default in MySQL 5.7.7 and later, the server iterates only the oldest and the newest binary log files and the values of gtid_purged and gtid_executed are computed based only on Previous_gtids_log_event or Gtid_log_event found in these files. This ensures only two binary log files are iterated during server restart or when binary logs are being purged.
数据库服务启动时，gtid_executed和gtid_purged按下面方式初始化
binlog_gtid_simple_recovery=FALSE
• gtid_executed：从mysql-bin.index的末行往首行所对应的binlog查找，直到首个被找到包含Previous_gtids_log_event的binlog。然后读取这个binlog的Previous_gtids_log_event和Gtid_log_events中的所有Gtid集合保存到内部变量gtids_in_binlog。然后使用gtids_in_binlog和mysql.gtid_executed表的并集初始化gtid_executed变量
如果你有大量非GTID的binlog(比如gtid_mode=off的情况下创建)，初始化gtid_executed的过程会消耗较长的时间
• gtid_purged：从mysql-bin.index的首行往末行所对应的binlog查找，直到首个被找到包含非空Previous_gtids_log_event或者Gtid_log_event的binlog。然后读取这个binlog的Previous_gtids_log_event，将gtids_in_binlog - Previous_gtids_log_event得到的集合保存到内部变量gtids_in_binlog_not_purged。最后使用gtid_executed - gtids_in_binlog_not_purged初始化gtid_purged变量
binlog_gtid_simple_recovery=TRUE(MySQL5.7.7及以上默认)
只迭代mysql-bin.index的首行和末行所对应的binlog，gtid_executed和gtid_purged的值就是取这两个binlog中的Previous_gtids_log_event/Gtid_log_event计算，当然gtid_executed变量的值还要结合mysql.gtid_executed
1.3、mysql.gtid_executed
https://dev.mysql.com/doc/refman/5.7/en/replication-gtids-concepts.html#replication-gtids-gtid-executed-table
GTIDs are stored in the mysql.gtid_executed table only when gtid_mode is ON or ON_PERMISSIVE. The point at which GTIDs are stored depends on whether binary logging is enabled or disabled:
• If binary logging is disabled (log_bin is OFF), or if log_slave_updates is disabled, the server stores the GTID belonging to each transaction together with the transaction in the table. In addition, the table is compressed periodically at a user-configurable rate; see mysql.gtid_executed Table Compression, for more information. This situation can only apply on a replication slave where binary logging or slave update logging is disabled. It does not apply on a replication master, because on a master, binary logging must be enabled for replication to take place.
• If binary logging is enabled (log_bin is ON), whenever the binary log is rotated or the server is shutdown, the server writes GTIDs for all transactions that were written into the previous binary log into the mysql.gtid_executed table. This situation applies on a replication master, or a replication slave where binary logging is enabled.
In the event of the server stopping unexpectedly, the set of GTIDs from the current binary log is not saved in the mysql.gtid_executed table. In this case, these GTIDs are added to the table and to the set of GTIDs in the gtid_executed system variable during recovery.
When binary logging is enabled, the mysql.gtid_executed table does not provide a complete record of the GTIDs for all executed transactions. That information is provided by the global value of the gtid_executed system variable.
如果没有开启log_bin或者没有开启log_slave_updates，从库在应用relay-log中的每个事务会执行一个insert mysql.gtid_executed操作。这只针对从库而言~
如果开启log_bin，在binlog发生rotate(flush binary logs/达到max_binlog_size)或者关闭服务时，会把所有写入到binlog中的Gtid信息写入到mysql.gtid_executed表。这适用于主库和从库~
log_bin=on，MySQL 8.0.17 起每个事务提交时会更新mysql.gtid_executed表
例子：从库log_bin=on，log_slave_updates=off，那么在应用relay-log时会实时写入mysql.gtid_executed，而在从库直接写入数据，需要等到发生rotate或者关闭服务才写入~
mysql.gtid_executed压缩：log-bin=off，每gtid_executed_compression_period压缩一次；log-bin=on，日志切换时压缩~
如果发生异常crash，当前binlog中的Gtids信息没能写入到mysql.gtid_executed表，在恢复过程通过读取binlog中的Previous_gtids_log_event/Gtid_log_event信息把这些Gtids添加到mysql.gtid_executed表和gtid_executed系统变量
二、实验
基本环境：官方社区版MySQL 5.7.19
[mysqld]
gtid-mode = on
binlog_gtid_simple_recovery = true
log-bin = /data/mysql/mysql3306/logs/mysql-bin
2.1、the oldest and newest file
前面初始化gtid_executed和gtid_purged时需迭代the newest和the oldest file. 最新和最旧分别对应的是哪个binary log？
2.1.1、binary log与index file一致
# 置空所有Gtid信息
mydba@192.168.85.132,3308 [(none)]> reset master;
Query OK, 0 rows affected (0.04 sec)
# 关闭服务
mydba@192.168.85.132,3308 [(none)]> shutdown; # 将另一实例下的binlog拷贝到当前实例的/data/mysql/mysql3308/logs/目录，并修改文件属主 # binlog列表
[root@ZST1 logs]# ll
total 32
-rw-r-----. 1 mysql mysql 2012 Jan 19 15:33 mysql-bin.000202
-rw-r-----. 1 mysql mysql 1528 Jan 19 15:33 mysql-bin.000203
-rw-r-----. 1 mysql mysql 545 Jan 19 15:33 mysql-bin.000204
-rw-r-----. 1 mysql mysql 873 Jan 19 15:33 mysql-bin.000205
-rw-r-----. 1 mysql mysql 217 Jan 19 15:33 mysql-bin.000206
-rw-r-----. 1 mysql mysql 1056 Jan 19 15:33 mysql-bin.000207
-rw-r-----. 1 mysql mysql 545 Jan 19 15:33 mysql-bin.000208
-rw-r--r--. 1 mysql mysql 308 Jan 19 15:40 mysql-bin.index
[root@ZST1 logs]# cat mysql-bin.index
/data/mysql/mysql3308/logs/mysql-bin.000202
/data/mysql/mysql3308/logs/mysql-bin.000203
/data/mysql/mysql3308/logs/mysql-bin.000204
/data/mysql/mysql3308/logs/mysql-bin.000205
/data/mysql/mysql3308/logs/mysql-bin.000206
/data/mysql/mysql3308/logs/mysql-bin.000207
/data/mysql/mysql3308/logs/mysql-bin.000208
[root@ZST1 logs]#
# 启动后登录
mydba@192.168.85.132,3308 [(none)]> show binary logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000202 | 2012 |
| mysql-bin.000203 | 1528 |
| mysql-bin.000204 | 545 |
| mysql-bin.000205 | 873 |
| mysql-bin.000206 | 217 |
| mysql-bin.000207 | 1056 |
| mysql-bin.000208 | 545 |
| mysql-bin.000209 | 194 |
+------------------+-----------+
8 rows in set (0.00 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507528 |
+--------------------------------------+----------------+--------------+
1 row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132,3308 [(none)]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507528 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged';
+---------------+-----------------------------------------------+
| Variable_name | Value |
+---------------+-----------------------------------------------+
| gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507510 |
+---------------+-----------------------------------------------+
1 row in set (0.04 sec)
解析mysql-bin.000208、mysql-bin.000202
# 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
...
# :: server id end_log_pos CRC32 0xed94846f Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
# :: server id end_log_pos CRC32 0x8b964e9a GTID last_committed= sequence_number= rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '8ab82362-9c37-11e7-a858-000c29c1025c:507528'/*!*/;
# at
# :: server id end_log_pos CRC32 0x15e5bb70 Query thread_id= exec_time= error_code=
SET TIMESTAMP=/*!*/;
BEGIN
/*!*/;
# at
# :: server id end_log_pos CRC32 0x16207d9b Table_map: `replcrash`.`py_user` mapped to number
# at
# :: server id end_log_pos CRC32 0x4fedbc9d Write_rows: table id flags: STMT_END_F
### INSERT INTO `replcrash`.`py_user`
### SET
### @= /* INT meta=0 nullable=0 is_null=0 */
### @='d6366f0e-fcb9-11e7-ad55-000c29' /* VARSTRING(96) meta=96 nullable=1 is_null=0 */
### @='2018-01-19 09:41:15' /* DATETIME(0) meta=0 nullable=1 is_null=0 */
### @='' /* VARSTRING(30) meta=30 nullable=1 is_null=0 */
# at
# :: server id end_log_pos CRC32 0x46352a20 Xid =
COMMIT/*!*/;
# at
# :: server id end_log_pos CRC32 0x6e53eee8 Stop
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@ZST1 logs]# gtids_in_binlog：[-]
mysql.gtid_executed：空
gtid_executed变量：gtids_in_binlog ∪ mysql.gtid_executed = [-] # 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xbb10fc9f Start: binlog v , server v 5.7.-log created :: at startup
ROLLBACK/*!*/;
# at
# :: server id end_log_pos CRC32 0x1e8fe6f0 Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
... gtids_in_binlog：[-]
oldest Previous_gtids_log_event：[-]
gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [-] - [-] = [-]
gtid_purged变量：gtid_executed - gtids_in_binlog_not_purged = [-] - [-] = [-]
binlog解析得到的数值和库中查询结果一致~
2.1.2、binary log与index file不一致
# 置空所有Gtid信息
mydba@192.168.85.132,3308 [(none)]> reset master;
Query OK, 0 rows affected (0.04 sec)
# 关闭服务
mydba@192.168.85.132,3308 [(none)]> shutdown; # 将另一实例下的binlog拷贝到当前实例的/data/mysql/mysql3308/logs/目录，并修改文件属主
# 并调整mysql-bin.index中的顺序 # binlog列表
[root@ZST1 logs]# ll
total 32
-rw-r-----. 1 mysql mysql 2012 Jan 19 17:11 mysql-bin.000202
-rw-r-----. 1 mysql mysql 1528 Jan 19 17:11 mysql-bin.000203
-rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000204
-rw-r-----. 1 mysql mysql 873 Jan 19 17:11 mysql-bin.000205
-rw-r-----. 1 mysql mysql 217 Jan 19 17:11 mysql-bin.000206
-rw-r-----. 1 mysql mysql 1056 Jan 19 17:11 mysql-bin.000207
-rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000208
-rw-r-----. 1 mysql mysql 308 Jan 19 17:12 mysql-bin.index
[root@ZST1 logs]# cat mysql-bin.index
/data/mysql/mysql3308/logs/mysql-bin.000203
/data/mysql/mysql3308/logs/mysql-bin.000202
/data/mysql/mysql3308/logs/mysql-bin.000204
/data/mysql/mysql3308/logs/mysql-bin.000205
/data/mysql/mysql3308/logs/mysql-bin.000206
/data/mysql/mysql3308/logs/mysql-bin.000208
/data/mysql/mysql3308/logs/mysql-bin.000207
[root@ZST1 logs]#
# 启动后登录
mydba@192.168.85.132,3308 [(none)]> show binary logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000203 | 1528 |
| mysql-bin.000202 | 2012 |
| mysql-bin.000204 | 545 |
| mysql-bin.000205 | 873 |
| mysql-bin.000206 | 217 |
| mysql-bin.000208 | 545 |
| mysql-bin.000207 | 1056 |
| mysql-bin.000209 | 194 |
+------------------+-----------+
8 rows in set (0.00 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507527 |
+--------------------------------------+----------------+--------------+
1 row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132,3308 [(none)]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507527 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged';
+---------------+-----------------------------------------------+
| Variable_name | Value |
+---------------+-----------------------------------------------+
| gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507517 |
+---------------+-----------------------------------------------+
1 row in set (0.01 sec)
解析mysql-bin.000207、mysql-bin.000203
# 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
...
COMMIT/*!*/;
# at
# :: server id end_log_pos CRC32 0x91996947 GTID last_committed= sequence_number= rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '8ab82362-9c37-11e7-a858-000c29c1025c:507527'/*!*/;
# at
# :: server id end_log_pos CRC32 0x41cdb52a Query thread_id= exec_time= error_code=
SET TIMESTAMP=/*!*/;
BEGIN
/*!*/;
# at
# :: server id end_log_pos CRC32 0xfcf11bdb Table_map: `replcrash`.`py_user` mapped to number
# at
# :: server id end_log_pos CRC32 0x0550d2ab Write_rows: table id flags: STMT_END_F
### INSERT INTO `replcrash`.`py_user`
### SET
### @= /* INT meta=0 nullable=0 is_null=0 */
### @='d5986a46-fcb7-11e7-ad55-000c29' /* VARSTRING(96) meta=96 nullable=1 is_null=0 */
### @='2018-01-19 09:26:55' /* DATETIME(0) meta=0 nullable=1 is_null=0 */
### @='' /* VARSTRING(30) meta=30 nullable=1 is_null=0 */
# at
# :: server id end_log_pos CRC32 0x15796ccd Xid =
COMMIT/*!*/;
# at
# :: server id end_log_pos CRC32 0xaf26375b Rotate to mysql-bin. pos:
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@ZST1 logs]# gtids_in_binlog：[-]
mysql.gtid_executed：空
gtid_executed变量：gtids_in_binlog ∪ mysql.gtid_executed = [-] # 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xd61e82fe Start: binlog v , server v 5.7.-log created :: at startup
ROLLBACK/*!*/;
# at
# :: server id end_log_pos CRC32 0x962d8c48 Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
... gtids_in_binlog：[-]
oldest Previous_gtids_log_event：[-]
gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [-] - [-] = [-]
gtid_purged变量：gtid_executed - gtids_in_binlog_not_purged = [-] - [-] = [-]
binlog解析得到的数值和库中查询结果一致~
上面的结果说明mysql.index中的首行和末行对应的就是oldest和newest
2.2、gtid_executed是来自gtids_in_binlog和mysql.gtid_executed的并集
前面的例子在服务启动时，读取最新的binlog中的Previous_gtids_log_event/Gtid_log_event，使用这些Gtid信息初始化gtid_executed，并将其写入到mysql.gtid_executed表
此时不再重置Gtid信息，而是继续缩小mysql-bin.index首行和末行的范围
# 关闭服务
mydba@192.168.85.132,3308 [(none)]> shutdown; # 将另一实例下的binlog拷贝到当前实例的/data/mysql/mysql3308/logs/目录，并修改文件属主
# 调整mysql-bin.index文件中列表顺序 # binlog列表
[root@ZST1 logs]# ll
total 32
-rw-r-----. 1 mysql mysql 2012 Jan 19 17:11 mysql-bin.000202
-rw-r-----. 1 mysql mysql 1528 Jan 19 17:11 mysql-bin.000203
-rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000204
-rw-r-----. 1 mysql mysql 873 Jan 19 17:11 mysql-bin.000205
-rw-r-----. 1 mysql mysql 217 Jan 19 17:11 mysql-bin.000206
-rw-r-----. 1 mysql mysql 1056 Jan 19 17:11 mysql-bin.000207
-rw-r-----. 1 mysql mysql 545 Jan 19 17:11 mysql-bin.000208
-rw-r-----. 1 mysql mysql 308 Jan 19 17:31 mysql-bin.index
[root@ZST1 logs]# cat mysql-bin.index
/data/mysql/mysql3308/logs/mysql-bin.000204
/data/mysql/mysql3308/logs/mysql-bin.000203
/data/mysql/mysql3308/logs/mysql-bin.000202
/data/mysql/mysql3308/logs/mysql-bin.000205
/data/mysql/mysql3308/logs/mysql-bin.000207
/data/mysql/mysql3308/logs/mysql-bin.000208
/data/mysql/mysql3308/logs/mysql-bin.000206
[root@ZST1 logs]#
# 启动后登录
mydba@192.168.85.132,3308 [(none)]> show binary logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000204 | 545 |
| mysql-bin.000203 | 1528 |
| mysql-bin.000202 | 2012 |
| mysql-bin.000205 | 873 |
| mysql-bin.000207 | 1056 |
| mysql-bin.000208 | 545 |
| mysql-bin.000206 | 217 |
| mysql-bin.000209 | 194 |
+------------------+-----------+
8 rows in set (0.03 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132,3308 [(none)]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 8ab82362-9c37-11e7-a858-000c29c1025c | 1 | 507527 |
+--------------------------------------+----------------+--------------+
1 row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132,3308 [(none)]> show master status;
+------------------+----------+--------------+------------------+-----------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-----------------------------------------------+
| mysql-bin.000209 | 194 | | | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507527 |
+------------------+----------+--------------+------------------+-----------------------------------------------+
1 row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132,3308 [(none)]> show variables like 'gtid_purged';
+---------------+-------------------------------------------------------------+
| Variable_name | Value |
+---------------+-------------------------------------------------------------+
| gtid_purged | 8ab82362-9c37-11e7-a858-000c29c1025c:1-507521:507525-507527 |
+---------------+-------------------------------------------------------------+
1 row in set (0.00 sec)
解析mysql-bin.000206、mysql-bin.000204
# 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0x17238df5 Start: binlog v , server v 5.7.-log created :: at startup
ROLLBACK/*!*/;
# at
# :: server id end_log_pos CRC32 0x3c03e609 Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
# :: server id end_log_pos CRC32 0x53b3b407 Stop
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@ZST1 logs]# gtids_in_binlog：[-]
mysql.gtid_executed：[-]
gtid_executed变量：gtids_in_binlog ∪ mysql.gtid_executed = [-] # 解析mysql-bin.
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin. |more
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xee45d7bf Start: binlog v , server v 5.7.-log created ::
# at
# :: server id end_log_pos CRC32 0xf8024ebe Previous-GTIDs
# 8ab82362-9c37-11e7-a858-000c29c1025c:-
# at
... gtids_in_binlog：[-]
oldest Previous_gtids_log_event：[-]
gtids_in_binlog_not_purged：gtids_in_binlog - oldest Previous_gtids_log_event = [-] - [-] = [-]
gtid_purged变量：gtid_executed - gtids_in_binlog_not_purged = [-] - [-] = [-:-]
上面的结果说明gtid_executed变量取的是gtids_in_binlog和mysql.gtid_executed的并集，并不是单纯来自某一个值~
2.3、binlog在Rotate或者关闭服务时，会把Gtid信息写入到mysql.gtid_executed表
2.3.1、关闭服务
写入一批数据，关闭服务，删除binlog后再启动服务，查看mysql.gtid_executed表
# 置空所有Gtid信息
mydba@192.168.85.132, [replcrash]> reset master;
Query OK, rows affected (0.01 sec)
# 清空测试数据表
mydba@192.168.85.132, [replcrash]> truncate table py_user;
Query OK, rows affected (1.49 sec)
# 写入数据(执行两次)
mydba@192.168.85.132, [replcrash]> insert into py_user(name,server_id) select left(uuid(),),@@server_id;
Query OK, row affected (0.00 sec)
Records: Duplicates: Warnings:
# 查看表中数据
mydba@192.168.85.132, [replcrash]> select * from py_user;
+-----+--------------------------------+---------------------+-----------+
| uid | name | add_time | server_id |
+-----+--------------------------------+---------------------+-----------+
| | 5a57574a--11e8--000c29 | -- :: | |
| | 65e773f6--11e8--000c29 | -- :: | |
+-----+--------------------------------+---------------------+-----------+
rows in set (0.00 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132, [replcrash]> select * from mysql.gtid_executed;
Empty set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132, [replcrash]> show master status;
+------------------+----------+--------------+------------------+------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+------------------------------------------+
| mysql-bin. | | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+------------------+----------+--------------+------------------+------------------------------------------+
row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132, [replcrash]> show variables like 'gtid_purged';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_purged | |
+---------------+-------+
row in set (0.00 sec)
# 关闭服务
mydba@192.168.85.132, [replcrash]> shutdown; # 此时的binlog
[root@ZST1 logs]# pwd
/data/mysql/mysql3306/logs
[root@ZST1 logs]# ll
total
-rw-r----- mysql mysql Jan : mysql-bin.
-rw-r----- mysql mysql Jan : mysql-bin.index
[root@ZST1 logs]# cat mysql-bin.index
/data/mysql/mysql3306/logs/mysql-bin.
[root@ZST1 logs]#
mysql-bin. 记录 60863f8d-01af-11e8-bfdf-000c29c1025c:- 的信息
# 删除binlog
[root@ZST1 logs]# cp -r ../logs ../logs_bak
[root@ZST1 logs]# rm -rf * # 启动服务
[root@ZST1 logs]# sh ~/start3306.sh
# 查看mysql.gtid_executed
mydba@192.168.85.132, [replcrash]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 60863f8d-01af-11e8-bfdf-000c29c1025c | | |
+--------------------------------------+----------------+--------------+
row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132, [replcrash]> show master status;
+------------------+----------+--------------+------------------+------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+------------------------------------------+
| mysql-bin. | | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+------------------+----------+--------------+------------------+------------------------------------------+
row in set (0.00 sec)
# 查看gtid_purged
mydba@192.168.85.132, [replcrash]> show variables like 'gtid_purged';
+---------------+------------------------------------------+
| Variable_name | Value |
+---------------+------------------------------------------+
| gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+---------------+------------------------------------------+
row in set (0.01 sec)
重启前mysql.gtid_executed的Gtid为空，关闭服务后删除所有binlog，重启后mysql.gtid_executed的Gtid:1-3。它的信息不可能来自binlog，只有可能在关闭服务时Gtid已写入到mysql.gtid_executed
2.3.2、flush binary logs
写入一批数据，flush binary logs，查看mysql.gtid_executed表，并解析binlog日志
# 写入数据(执行两次)
mydba@192.168.85.132, [replcrash]> insert into py_user(name,server_id) select left(uuid(),),@@server_id;
Query OK, row affected (0.04 sec)
Records: Duplicates: Warnings:
# 查看mysql.gtid_executed
mydba@192.168.85.132, [replcrash]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 60863f8d-01af-11e8-bfdf-000c29c1025c | | |
+--------------------------------------+----------------+--------------+
row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132, [replcrash]> show master status;
+------------------+----------+--------------+------------------+------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+------------------------------------------+
| mysql-bin. | | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+------------------+----------+--------------+------------------+------------------------------------------+
row in set (0.00 sec)
# 查看mysql.gtid_purged
mydba@192.168.85.132, [replcrash]> show variables like 'gtid_purged';
+---------------+------------------------------------------+
| Variable_name | Value |
+---------------+------------------------------------------+
| gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+---------------+------------------------------------------+
row in set (0.01 sec)
写入数据，mysql.gtid_executed表并不会时时更新 # 切换日志
mydba@192.168.85.132, [replcrash]> flush binary logs;
Query OK, rows affected (0.01 sec)
# 查看mysql.gtid_executed
mydba@192.168.85.132, [replcrash]> select * from mysql.gtid_executed;
+--------------------------------------+----------------+--------------+
| source_uuid | interval_start | interval_end |
+--------------------------------------+----------------+--------------+
| 60863f8d-01af-11e8-bfdf-000c29c1025c | | |
+--------------------------------------+----------------+--------------+
row in set (0.00 sec)
# 查看当前位置
mydba@192.168.85.132, [replcrash]> show master status;
+------------------+----------+--------------+------------------+------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+------------------------------------------+
| mysql-bin. | | | | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+------------------+----------+--------------+------------------+------------------------------------------+
row in set (0.00 sec)
# 查看mysql.gtid_purged
mydba@192.168.85.132, [replcrash]> show variables like 'gtid_purged';
+---------------+------------------------------------------+
| Variable_name | Value |
+---------------+------------------------------------------+
| gtid_purged | 60863f8d-01af-11e8-bfdf-000c29c1025c:- |
+---------------+------------------------------------------+
row in set (0.01 sec)
flush logs后gtid_purged变量的值马上更新到mysql.gtid_executed表
mysql.gtid_executed表并不是实时更新，flush logs后gtid_executed变量的值马上更新到mysql.gtid_executed表
解析binlog日志
# 解析切换后的日志
[root@ZST1 logs]# ll
total
-rw-r----- mysql mysql Jan : mysql-bin.
-rw-r----- mysql mysql Jan : mysql-bin.
-rw-r----- mysql mysql Jan : mysql-bin.index
[root@ZST1 logs]# mysqlbinlog -vv --base64-output=decode-rows mysql-bin.
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at
# :: server id end_log_pos CRC32 0xea188782 Start: binlog v , server v 5.7.-log created ::
# Warning: this binlog is either in use or was not closed properly.
# at
# :: server id end_log_pos CRC32 0xdf719f1d Previous-GTIDs
# 60863f8d-01af-11e8-bfdf-000c29c1025c:-
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@ZST1 logs]#
注意Rotate后的binlog中Previous-GTIDs是4-5 ，而不是1-5。因为Rotate只把binary log中的Gtid写入到新binary log，同时更新mysql.gtid_executed表
2.4、如何通过binlog和mysql.gtid_executed得到gtid_executed、gtid_purged
搞清楚服务启动前mysql.gtid_executed的值。对于5.7.7及以后默认只需迭代最新和最旧binlog中的Previous_gtids_log_event/Gtid_log_event，计算gtids_in_binlog、gtids_in_binlog_not_purged，然后根据公式计算gtid_executed、gtid_purged变量。可以查看前面解析binlog的例子
三、参考文档
Global Transaction ID Options and Variables：https://dev.mysql.com/doc/refman/5.7/en/replication-options-gtids.html
MySQL 5.7 Gtid内部学习(五) mysql.gtid_executed表/gtid_executed变量/gtid_purged变量的更改时机：https://yq.aliyun.com/articles/294004
[MySQL 5.6] GTID实现、运维变化及存在的bug：http://www.cnblogs.com/MYSQLZOUQI/p/3850578.html
binlog rotate引发的MySQL阻塞事件：https://mp.weixin.qq.com/s/XSnFkuYzIlGWMaXIl-oPeQ
gtid_executed和gtid_purged变量是如何初始化的的更多相关文章
- MySQL Replication--全局参数gtid_executed和gtid_purged
参数定义gtid_executed,全局参数,GTID集合包含所有在该服务器上执行过的事务编号和使用set gtid_purged语句设置过的事务编号,使用SHOW MASTER STATUS和SHO ...
- 关于Javascript循环体变量声明与初始化的效率问题
针对循环体变量声明与初始化的效率问题,将执行的简单测试代码如下: function test(n) { console.time('Internally initialized'); for (var ...
- C++变量的默认初始化规则
定义没有初始化式的变量时,系统有时候会帮我们初始化变量.系统如何初始化取决于变量的类型以及变量定义的位置. 内置类型变量是否自动初始化取决于变量定义的位置.函数体外定义的变量初始成0:函数体内定义的变 ...
- GoLang学习之变量定义和初始化
变量命名原则 go语言的变量名有字母数字和下划线组成,首字母不能为数字,但是字母不仅仅只限于英文字母,所有的UTF-8字符都是可以的. 变量声明和初始化方式 使用var关键字 var a int = ...
- Java初始化顺序（静态变量、静态初始化块、实例变量、实例初始化块、构造方法）
1.执行顺序 1.1.一个类中的初始化顺序 类内容(静态变量.静态初始化块) => 实例内容(变量.初始化块.构造器) 1.2.两个具有继承关系类的初始化顺序 父类的(静态变量.静态初始化块)= ...
- 重要：C/C++变量的自动初始化
对于内置变量的自动初始化 代码1: #include<stdio.h> #define CONST 100 int *p1; ]; int b; static int c; main() ...
- Java类的初始化顺序 （静态变量、静态初始化块、变量、初始...
很有意思的一篇文章 1.没有继承 静态变量->静态初始化块->变量->变量初始化块->构造方法 2.有继承的情况 父类静态变量->父类静态初始化块->子类静态变量- ...
- 【C++】const，static和static const类型成员变量声明及其初始化
1)const定义的常量在超出其作用域之后其空间会被释放,而static定义的静态常量在函数执行后不会释放其存储空间 void f1() { ; cout<<x<<endl; ...
- QT，静态变量要记得初始化
//DbUtil.h #ifndef DBUTIL_H #define DBUTIL_H using namespace std; QString md5Encode(QString passwd); ...
随机推荐
- 核化主成分分析（Kernel PCA）应用及调参
核化这个概念在很多机器学习方法中都有应用,如SVM,PCA等.在此结合sklearn中的KPCA说说核函数具体怎么来用. KPCA和PCA都是用来做无监督数据处理的,但是有一点不一样.PCA是降维,把 ...
- Socket网络编程--Libev库学习(1)
这一节是安装篇. Socket网络编程不知不觉已经学了快两个月了.现在是时候找个网络库学学了.搜索了很多关于如何学网络编程的博客和问答.大致都是推荐学一个网络库,至于C++网络库有那么几个,各有各的好 ...
- File操作
对文件进行操作(只操作小文件) bool Exists(string path) 判断文件是否存在 FileStream Create(string path) 创建文件 void Move(stri ...
- ceph 搭建nginx负载3个对象网关
nginx.conf user nginx; worker_processes auto; error_log /var/log/nginx/error.log; pid /run/nginx.pid ...
- 自学Linux Shell14.1-理解输入输出
点击返回 自学Linux命令行与Shell脚本之路 14.1-理解输入输出(点击这里获得更多) 两种显示脚本输出的方法: 在显示器屏幕上显示输出 将输出重定向到文件中 1. 标准文件描述符 linux ...
- UOJ #314. 【NOI2017】整数 | 线段树 压位
题目链接 UOJ 134 题解 可爱的电音之王松松松出的题--好妙啊. 首先想一个朴素的做法! 把当前的整数的二进制当作01序列用线段树维护一下(序列的第i位就是整数中位权为\(2^k\)的那一位). ...
- Mysql插入、替换和无视
在现实的生产环境下,通常我们都会遇到这样的应用场景,需要我们更新数据.替换数据,若存在则不操作. insert into表示插入数据,数据库会检查主键,如果出现重复会报错: replace into表 ...
- eclipse index 不工作 F3 不能找到头文件
To add paths containing code to parse, follow these steps :1. Right click on the project2. Select Pr ...
- Xshell配置是vi显示多种颜色
在链接中,点 File菜单——properties 或按快捷键 alt+p 第一步: Properties--->Terminal 右边的窗口中,将Terminal Type 选择为linu ...
- Visio画UML类图、序列图 for Java
参考文档: 1.百度搜索: 怎样用Visio 2007画C++类图 连接 https://jingyan.baidu.com/article/9f7e7ec07286e16f281554f7.html ...