MySQL Point in Time Recovery the Right Way
In this blog, I’ll look at how to do MySQL point in time recovery (PITR) correctly.
Sometimes we need to restore from a backup, and then replay the transactions that happened after the backup was taken. This is a common procedure in most disaster recovery plans, when for example you accidentally drop a table/database or run an update/delete without the “where” clause and lose data.
The usual way is to get a copy of your binlogs and use mysqlbinlog to replay those transactions. But this approach has many pitfalls that can make the whole PITR process a nightmare. Some examples:
- You need to make sure to run a single mysqlbinlog command with all related binlogs, and pipe them to mysql at once. Otherwise, if binlog.000001 creates a temporary table, and binlog.000002 requires that temporary table, it will not be present. Each execution of MySQL creates a new connection:
Shell
|
1
2
|
shell> mysqlbinlog binlog.000001 | mysql -u root -p # Creates tmp table X
shell> mysqlbinlog binlog.000002 | mysql -u root -p # Uses tmp table X
|
- We can say that it has to be an atomic operation. If it fails halfway through, it will be very difficult to know where it failed and even more difficult to resume from that point forward. There are many reasons for it to fail: InnoDB lock wait timeout / deadlock with some concurrent transaction, server and client have differentmax_allowed_packet and you get a
Lost connection to MySQL server during queryerror, and so on.
So how can we overcome those limitations and have a reliable way to do Point In Time Recovery?
We can restore the backup on the desired server, build a second server with just the minimal MySQL required data and move the all binary logs to this “fake” server datadir. Then we need to configure the server where we want the PITR to happen as a slave of the fake server. From this point forward, it’s going to be pure MySQL replication happening.
To illustrate it, I create a Docker container on the machine. I have Percona Server for MySQL running on the box listening on 3306, and have already restored the backup on it. There is a tarball there with all binlogs required. The saved positions for PITR are as follows:
Shell
|
1
2
|
[root@localhost ~]# cat /var/lib/mysql/xtrabackup_binlog_info
master-bin.000007 1518932
|
I create a folder to store the Docker MySQL datadir:
Shell
|
1
2
|
mkdir /tmp/pitr
chown -R 1001 /tmp/pitr
|
I start the Docker container. As we can see from xtrabackup_binlog_info, my binlogs are named master-bin and I’ll be setting the same server-id as original master:
Shell
|
1
2
3
4
|
docker run --name ps_pitr -v /tmp/pitr:/var/lib/mysql
-p 3307:3306 -e MYSQL_ROOT_PASSWORD=secret
-d percona/percona-server:5.7.18
--log_bin=master-bin --server-id=10
|
In case you want to make usage of GTID, append --gtid-mode=ON --enforce_gtid_consistency=ON to the end of the Docker command.
The command above starts a MySQL instance, invokes mysqld –initialize, sets the root password to secret and it’s port 3306 is mapped back to my local 3307 port. Now I’ll stop it, remove the binlogs that it created, uncompress and move all required binlogs to its datadir mapped folder and start it again:
Shell
|
1
2
3
4
5
|
docker stop ps_pitr
rm /tmp/pitr/master-bin.*
tar -zxf binlogs.tgz -C /tmp/pitr
chown -R 1001 /tmp/pitr/master-bin.*
docker start ps_pitr
|
If it all worked correctly, at this point we can see the full list of binary logs on the Docker container by connecting on port 3307:
Shell
|
1
2
3
4
5
6
7
8
9
10
11
|
mysql -u root -psecret -P 3307 --protocol=TCP -e "SHOW BINARY LOGS"
mysql: [Warning] Using a password on the command line interface can be insecure.
+-------------------+-----------+
| Log_name | File_size |
+-------------------+-----------+
| master-bin.000005 | 26216208 |
| master-bin.000006 | 26214614 |
| master-bin.000007 | 26214902 |
. . .
| master-bin.000074 | 154 |
+-------------------+-----------+
|
Now, all we need to do is connect to our server, which has the backup restored, and configure it as a slave from 3307:
Shell
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
|
mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or g.
Your MySQL connection id is 6
Server version: 5.7.18-16 Percona Server (GPL), Release 16, Revision d7301f8
Copyright (c) 2009-2017 Percona LLC and/or its affiliates
Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
mysql> CHANGE MASTER TO MASTER_HOST='127.0.0.1', MASTER_PORT=3307, MASTER_USER='root', MASTER_PASSWORD='secret', MASTER_LOG_FILE='master-bin.000007', MASTER_LOG_POS=1518932;
Query OK, 0 rows affected, 2 warnings (0.01 sec)
mysql> START SLAVE;
Query OK, 0 rows affected (0.01 sec)
mysql> SHOW SLAVE STATUSG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 127.0.0.1
Master_User: root
Master_Port: 3307
Connect_Retry: 60
Master_Log_File: master-bin.000008
Read_Master_Log_Pos: 449696
Relay_Log_File: localhost-relay-bin.000002
Relay_Log_Pos: 28957
Relay_Master_Log_File: master-bin.000007
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 15217950
Relay_Log_Space: 11476311
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 4382
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 10
Master_UUID: 80b9fe26-a945-11e7-aa1d-0242ac110002
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Opening tables
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.17 sec)
. . .
mysql> SHOW SLAVE STATUSG
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 127.0.0.1
Master_User: root
Master_Port: 3307
Connect_Retry: 60
Master_Log_File: master-bin.000074
Read_Master_Log_Pos: 154
Relay_Log_File: localhost-relay-bin.000133
Relay_Log_Pos: 381
Relay_Master_Log_File: master-bin.000074
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 154
Relay_Log_Space: 819
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 10
Master_UUID: 80b9fe26-a945-11e7-aa1d-0242ac110002
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.01 sec)
|
If you want to apply logs up to a particular time you can make use of mysqlbinlog to verify what the last position / GTID it should apply, and use START SLAVE UNTIL MASTER_LOG_FILE = 'log_name', MASTER_LOG_POS = log_pos or START SLAVE SQL_THREAD UNTIL SQL_AFTER_GTIDS = 3E11FA47-71CA-11E1-9E33-C80AA9429562:11-56.
Special thanks to Marcos Albe, who originally showed me this MySQL point in time recovery approach
MySQL Point in Time Recovery the Right Way的更多相关文章
- [转载] 淘宝内部分享：怎么跳出MySQL的10个大坑（上)
原文: http://mp.weixin.qq.com/s?__biz=MzAxNjAzMTQyMA==&mid=209773318&idx=1&sn=e9600d3db80b ...
- 怎么跳出MySQL的10个大坑
淘宝自从2010开始规模使用MySQL,替换了之前商品.交易.用户等原基于IOE方案的核心数据库,目前已部署数千台规模.同时和Oracle, Percona, Mariadb等上游厂商有良好合作,共向 ...
- 淘宝内部分享：怎么跳出MySQL的10个大坑
编者按:淘宝自从2010开始规模使用MySQL,替换了之前商品.交易.用户等原基于IOE方案的核心数据库,目前已部署数千台规模.同时和Oracle, Percona, Mariadb等上游厂商有良好合 ...
- mysql生成数据字典
git clone https://github.com/twindb/undrop-for-innodb.git make [root@redis01 undrop-for-innodb]# mak ...
- [MySQL] xtrabakcup原理
Xtrabackup InnoDB内部的Redo log, 也叫Transaction log file. 存储每一个InnoDB表纪录的修改日志. 当InnoDB启动时, InnoDB会检查数据文件 ...
- 利用 force recovery 解决服务器 crash 导致 MySQL 重启失败的问题
小明同学在本机上安装了 MySQL 5.7.17 配合项目进行开发,并且已经有了一部分重要数据.某天小明在开发的时候,需要出去一趟就直接把电脑关掉了,没有让 MySQL 正常关闭,重启 MySQL 的 ...
- 利用 Forcing InnoDB Recovery 特性解决 MySQL 重启失败的问题
小明同学在本机上安装了 MySQL 5.7.17 配合项目进行开发,并且已经有了一部分重要数据.某天小明在开发的时候,需要出去一趟就直接把电脑关掉了,没有让 MySQL 正常关闭,重启 MySQL 的 ...
- MySQL InnoDB Update和Crash Recovery流程
MySQL InnoDB Update和Crash Recovery流程 概要信息 首先介绍了Redo,Undo,Log Sequence Number (LSN),Checkpoint,Rollba ...
- mysql中的备份（backup）和恢复（recovery）
(一)备份类型(backup type) 物理和逻辑备份(Physical Versus Logical Backup) 物理备份是指直接复制存储数据库内容的目录和文件,这种类型的备份适用于出现问题时 ...
随机推荐
- npm下设置NODE_ENV下mac与windows的差异
linux & mac: export NODE_ENV=production windows:set NODE_ENV=production windows: mac:
- codeforces727E. Games on a CD
题意:给一个n*k的循环字符串可能从任意地方断开,然后m个长度k的字符串,问你能不能用下面的字符串(每个最多用一次)构成上面的字符串,能循环移位 题解:对下面的串建ac自动机,记录字符串最后一个位置, ...
- Matlab:椭圆方程的导数边值问题
tic; clear clc N=; M=*N; h1=/M; h2=/N; x=:h1:; y=:h2:; fun=inline('exp(x)*sin(pi*y)','x','y'); f=inl ...
- MongoDB学习笔记——数据库的创建与初始
Part1:MongoDB与SQL的概念对比 图片来源--菜鸟教程 Part2:MongoDB安装地址 直接下载地址:戳这里 备用地址:戳这里 通过备用地址(官网)下载时,要注意下面这个地方 Part ...
- iTerm2 + oh my zsh +agnoster 打造最强Mac终端
链接: http://www.siguoya.name/pc/home/article/256 http://www.jianshu.com/p/fabd40cf83fe http://www.jia ...
- Excel中针对IP地址的排序方法
新建一个辅助排序列,用辅助列来扩展,辅助列公式如下: =TRIM(TEXT(LEFT(SUBSTITUTE(A1,".",REPT(" ",99)),100), ...
- ES6--闭包数组i的值与var的作用域理解
var a = [];for (var i = 0; i < 10; i++) { a[i] = function () { console.log(i); };}a[6](); // 10 变 ...
- Vue(四) 内置指令
现在介绍 Vue.js 中 更多的内置指令 基本指令 v-cloak v-cloak 不需要表达式,它会在 Vue 实例结束编译时从绑定的 HTML 元素上移除,经常和 CSS 的 display: ...
- [Codeforces771E]Bear and Rectangle Strips
Problem 给你一个2*n的矩阵,要求你用补充叠的矩阵去框,要求每个矩阵框中的数之和为0,问最多可以用几个矩阵. Solution 首先预处理出一个点到离它最近的一段和为0的区间的左端点 然后到这 ...
- POST形式 soapUI调用WebService的restful接口，传入json参数，并且返回json
第一次使用POST形式传JSON字符串,怎么都调不到后台方法,只是因为注解没加对…… CXF的WebService接口类 package com.zit.webservice.main; import ...