mysql5.7 闪回数据（update delete insert）
本次测试用Myflash闪回dml操作，有个前提条件是log_bin开启并且log模式是row；
mysql> show global variables like "binlog%";
+--------------------------------------------+--------------+
| Variable_name | Value |
+--------------------------------------------+--------------+
| binlog_cache_size | |
| binlog_checksum | NONE |
| binlog_direct_non_transactional_updates | OFF |
| binlog_error_action | ABORT_SERVER |
| binlog_format | ROW |
| binlog_group_commit_sync_delay | |
| binlog_group_commit_sync_no_delay_count | |
| binlog_gtid_simple_recovery | ON |
| binlog_max_flush_queue_time | |
| binlog_order_commits | ON |
| binlog_row_image | FULL |
| binlog_rows_query_log_events | OFF |
| binlog_stmt_cache_size | |
| binlog_transaction_dependency_history_size | |
| binlog_transaction_dependency_tracking | COMMIT_ORDER |
+--------------------------------------------+--------------+
rows in set (0.00 sec) mysql> show variables like 'log_%';
+----------------------------------------+-----------------------------+
| Variable_name | Value |
+----------------------------------------+-----------------------------+
| log_bin | ON |
| log_bin_basename | /data/mysql/mysql-bin |
| log_bin_index | /data/mysql/mysql-bin.index |
| log_bin_trust_function_creators | OFF |
| log_bin_use_v1_row_events | OFF |
| log_builtin_as_identified_by_password | OFF |
| log_error | /data/log/mysql-error.log |
| log_error_verbosity | |
| log_output | FILE |
| log_queries_not_using_indexes | OFF |
| log_slave_updates | ON |
| log_slow_admin_statements | OFF |
| log_slow_slave_statements | OFF |
| log_statements_unsafe_for_binlog | ON |
| log_syslog | OFF |
| log_syslog_facility | daemon |
| log_syslog_include_pid | ON |
| log_syslog_tag | |
| log_throttle_queries_not_using_indexes | |
| log_timestamps | UTC |
| log_warnings | |
+----------------------------------------+-----------------------------+
rows in set (0.00 sec)
安装myflash
git clone https://github.com/Meituan-Dianping/MyFlash.git
myflash依赖glibc查询
[root@redis01 MyFlash]# rpm -qa|grep glibc
glibc-headers-2.12-1.209.0.1.el6.x86_64
glibc-2.12-1.209.0.1.el6.x86_64
glibc-common-2.12-1.209.0.1.el6.x86_64
glibc-devel-2.12-1.209.0.1.el6.x86_64 动态编译
gcc -w `pkg-config --cflags --libs glib-2.0` source/binlogParseGlib.c -o binary/flashback [root@redis01 binary]# ./flashback -h
Usage:
flashback [OPTION...] Help Options:
-h, --help Show help options Application Options:
--databaseNames databaseName to apply. if multiple, seperate by comma(,)
--tableNames tableName to apply. if multiple, seperate by comma(,)
--start-position start position
--stop-position stop position
--start-datetime start time (format %Y-%m-%d %H:%M:%S)
--stop-datetime stop time (format %Y-%m-%d %H:%M:%S)
--sqlTypes sql type to filter . support INSERT, UPDATE ,DELETE. if multiple, seperate by comma(,)
--maxSplitSize max file size after split, the uint is M
--binlogFileNames binlog files to process. if multiple, seperate by comma(,)
--outBinlogFileNameBase output binlog file name base
--logLevel log level, available option is debug,warning,error
--include-gtids gtids to process
--exclude-gtids gtids to skip
生成100万行数据
[root@redis01 sysbench]# sysbench oltp_common --tables=1 --table-size=1000000 --db-driver=mysql --mysql-db=ht --mysql-user=root --mysql-password=ocm123 prepare
sysbench 1.0.14 (using bundled LuaJIT 2.1.0-beta2) Creating table 'sbtest1'...
Inserting 1000000 records into 'sbtest1'
Creating a secondary index on 'sbtest1'...
查看binglog的pos
mysql> reset master;
Query OK, 0 rows affected (0.17 sec) mysql> show master logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000001 | 150 |
+------------------+-----------+
1 row in set (0.00 sec) mysql> show master logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000001 | 190803685 |
+------------------+-----------+
1 row in set (0.00 sec)
查看表中数据
mysql> select * from ht.sbtest1 where id < 10;
+----+--------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
| id | k | c | pad |
+----+--------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
| 1 | 499284 | 83868641912-28773972837-60736120486-75162659906-27563526494-20381887404-41576422241-93426793964-56405065102-33518432330 | 67847967377-48000963322-62604785301-91415491898-96926520291 |
| 2 | 501969 | 38014276128-25250245652-62722561801-27818678124-24890218270-18312424692-92565570600-36243745486-21199862476-38576014630 | 23183251411-36241541236-31706421314-92007079971-60663066966 |
| 3 | 504261 | 33973744704-80540844748-72700647445-87330233173-87249600839-07301471459-22846777364-58808996678-64607045326-48799346817 | 38615512647-91458489257-90681424432-95014675832-60408598704 |
| 4 | 502014 | 37002370280-58842166667-00026392672-77506866252-09658311935-56926959306-83464667271-94685475868-28264244556-14550208498 | 63947013338-98809887124-59806726763-79831528812-45582457048 |
| 5 | 499813 | 44257470806-17967007152-32809666989-26174672567-29883439075-95767161284-94957565003-35708767253-53935174705-16168070783 | 34551750492-67990399350-81179284955-79299808058-21257255869 |
| 6 | 500585 | 37216201353-39109531021-11197415756-87798784755-02463049870-83329763120-57551308766-61100580113-80090253566-30971527105 | 05161542529-00085727016-35134775864-52531204064-98744439797 |
| 7 | 501371 | 33071042495-29920376648-91343430102-79082003121-73317691963-02846712788-88069761578-14885283975-44409837760-90760298045 | 91798303270-64988107984-08161247972-12116454627-22996445111 |
| 8 | 499938 | 73754818686-04889373966-18668178968-56957589012-31352882173-91882653509-59577900152-88962682169-52981807259-62646890059 | 76460662325-41613089656-42706083314-81833284991-17063140920 |
| 9 | 527268 | 26482547570-00155460224-12388481921-23289186371-78242522654-77998886134-73270876420-50821093220-31442690639-11588920653 | 30508501104-50823269125-88107014550-70202920684-95842308929 |
+----+--------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
9 rows in set (0.00 sec)
更新行，并且查看binglog
mysql> update ht.sbtest1 set pad='qdds';
Query OK, 1000000 rows affected (33.62 sec)
Rows matched: 1000000 Changed: 1000000 Warnings: 0 mysql> select * from ht.sbtest1 where id < 10;
+----+--------+-------------------------------------------------------------------------------------------------------------------------+------+
| id | k | c | pad |
+----+--------+-------------------------------------------------------------------------------------------------------------------------+------+
| 1 | 499284 | 83868641912-28773972837-60736120486-75162659906-27563526494-20381887404-41576422241-93426793964-56405065102-33518432330 | qdds |
| 2 | 501969 | 38014276128-25250245652-62722561801-27818678124-24890218270-18312424692-92565570600-36243745486-21199862476-38576014630 | qdds |
| 3 | 504261 | 33973744704-80540844748-72700647445-87330233173-87249600839-07301471459-22846777364-58808996678-64607045326-48799346817 | qdds |
| 4 | 502014 | 37002370280-58842166667-00026392672-77506866252-09658311935-56926959306-83464667271-94685475868-28264244556-14550208498 | qdds |
| 5 | 499813 | 44257470806-17967007152-32809666989-26174672567-29883439075-95767161284-94957565003-35708767253-53935174705-16168070783 | qdds |
| 6 | 500585 | 37216201353-39109531021-11197415756-87798784755-02463049870-83329763120-57551308766-61100580113-80090253566-30971527105 | qdds |
| 7 | 501371 | 33071042495-29920376648-91343430102-79082003121-73317691963-02846712788-88069761578-14885283975-44409837760-90760298045 | qdds |
| 8 | 499938 | 73754818686-04889373966-18668178968-56957589012-31352882173-91882653509-59577900152-88962682169-52981807259-62646890059 | qdds |
| 9 | 527268 | 26482547570-00155460224-12388481921-23289186371-78242522654-77998886134-73270876420-50821093220-31442690639-11588920653 | qdds |
+----+--------+-------------------------------------------------------------------------------------------------------------------------+------+
9 rows in set (0.43 sec) mysql> show master logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000001 | 517083888 |
+------------------+-----------+
1 row in set (0.00 sec)
准备脚本闪回
[root@redis01 binary]# cat 1.sh
echo `date`
export MYSQL_PWD=ocm123
./flashback --databaseNames=ht --tableNames=sbtest1 --sqlTypes=update --binlogFileNames=/data/mysql/mysql-bin.000001 --outBinlogFileNameBase=./org --start-position=190803685
./flashback --binlogFileNames=org.flashback --maxSplitSize=30
arr=`find ${file_path} -name "binlog_output_base.0*"|sort -n`
for i in ${arr}
do
mysqlbinlog --skip-gtids ${i} | mysql ht
done
echo `date` [root@redis01 binary]# ./1.sh
Thu Jul 12 18:36:46 CST 2018
Thu Jul 12 18:42:29 CST 2018
闪回100万行数据只用6分钟
闪回完毕查看数据
mysql> select * from ht.sbtest1 where id < 10;
+----+--------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
| id | k | c | pad |
+----+--------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
| 1 | 499284 | 83868641912-28773972837-60736120486-75162659906-27563526494-20381887404-41576422241-93426793964-56405065102-33518432330 | 67847967377-48000963322-62604785301-91415491898-96926520291 |
| 2 | 501969 | 38014276128-25250245652-62722561801-27818678124-24890218270-18312424692-92565570600-36243745486-21199862476-38576014630 | 23183251411-36241541236-31706421314-92007079971-60663066966 |
| 3 | 504261 | 33973744704-80540844748-72700647445-87330233173-87249600839-07301471459-22846777364-58808996678-64607045326-48799346817 | 38615512647-91458489257-90681424432-95014675832-60408598704 |
| 4 | 502014 | 37002370280-58842166667-00026392672-77506866252-09658311935-56926959306-83464667271-94685475868-28264244556-14550208498 | 63947013338-98809887124-59806726763-79831528812-45582457048 |
| 5 | 499813 | 44257470806-17967007152-32809666989-26174672567-29883439075-95767161284-94957565003-35708767253-53935174705-16168070783 | 34551750492-67990399350-81179284955-79299808058-21257255869 |
| 6 | 500585 | 37216201353-39109531021-11197415756-87798784755-02463049870-83329763120-57551308766-61100580113-80090253566-30971527105 | 05161542529-00085727016-35134775864-52531204064-98744439797 |
| 7 | 501371 | 33071042495-29920376648-91343430102-79082003121-73317691963-02846712788-88069761578-14885283975-44409837760-90760298045 | 91798303270-64988107984-08161247972-12116454627-22996445111 |
| 8 | 499938 | 73754818686-04889373966-18668178968-56957589012-31352882173-91882653509-59577900152-88962682169-52981807259-62646890059 | 76460662325-41613089656-42706083314-81833284991-17063140920 |
| 9 | 527268 | 26482547570-00155460224-12388481921-23289186371-78242522654-77998886134-73270876420-50821093220-31442690639-11588920653 | 30508501104-50823269125-88107014550-70202920684-95842308929 |
+----+--------+-------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+
9 rows in set (0.00 sec)
mysql> show master logs;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000001 | 851563888 |
+------------------+-----------+
1 row in set (0.00 sec)
测试过程当中出现的问题
ERROR 1782 (HY000) at line 18: @@SESSION.GTID_NEXT cannot be set to ANONYMOUS when @@GLOBAL.GTID_MODE = ON.
解决：mysqlbinlog 加--skip-gtids
[root@redis01 binary]# mysqlbinlog --skip-gtids binlog_output_base.flashback | mysql ht
ERROR 2006 (HY000) at line 8017: MySQL server has gone away
解决：加大max_allowed_packet
mysql> show global variables like 'max_allowed_packet';
+--------------------+---------+
| Variable_name | Value |
+--------------------+---------+
| max_allowed_packet | 4194304 |
+--------------------+---------+
1 row in set (0.00 sec) set global max_allowed_packet=67108864;
还有用下面脚本执行完毕不回滚数据
./flashback --databaseNames=ht --tableNames=sbtest1 --sqlTypes=update --binlogFileNames=/data/mysql/mysql-bin.000001 --maxSplitSize=30 --start-position=190803685
mysqlbinlog --skip-gtids ${i} | mysql ht
疑惑的是直接切割生成的日志文件不能回滚数据；得用下面的脚本在单独其切割成小文件才能回滚数据；后面有空得看看源码尝试自己修改下
./flashback --databaseNames=ht --tableNames=sbtest1 --sqlTypes=update --binlogFileNames=/data/mysql/mysql-bin.000001 --outBinlogFileNameBase=./org --start-position=190803685
./flashback --binlogFileNames=org.flashback --maxSplitSize=30
mysql5.7 闪回数据（update delete insert）的更多相关文章
- [转]SQLServer添加UPDATE回滚日志(update/delete/insert)
下面直接上代码(copy到你的数据库里面直接就可以运行): CREATE PROCEDURE [dbo].[SP_UPDATE_LOG] ) AS BEGIN SET NOCOUNT ON; IF N ...
- oracle flashback data archive闪回数据归档天坑之XID重用导致闪回查询数据重复
我们有个系统使用了Oracle flashback data archive闪回数据归档特性来作为基于时间点的恢复机制,在频繁插入.更新期间发现SYS_FBA_HIST_NNNN表中的XID被两个事务 ...
- oracle闪回、闪回数据归档Flashback Data Archive (Oracle Total Recall)的真正强大之处、11gR2增强以及合理使用
oracle的闪回很早就出来了,准确的说一直以来应该都较少被真正用户广为使用,除了dba和极少部分开发人员偶尔用于逻辑出错.误删恢复之外,较少被用于产生更有价值的用途. 各种闪回表flashback ...
- flashback query闪回数据
误删除了部分重要数据,已提交,需要恢复.首先尝试flashback query闪回数据. 数据库运行在归档模式,首先确认数据库的SCN的变化: SQL> col fscn for 9999999 ...
- Oracle闪回查询恢复delete删除数据
Flashback query(闪回查询)原理 Oracle根据undo信息,利用undo数据,类似一致性读取方法,可以把表置于一个删除前的时间点(或SCN),从而将数据找回. Flashback q ...
- PLSQL_闪回删除FlashBack Delete表误删除如何进行恢复（案例）
2014-07-02 Created By BaoXinjian
- 除了binlog2sql工具外，使用python脚本闪回数据（数据库误操作）
利用binlog日志恢复数据库误操作数据 在人工手动进行一些数据库写操作的时候(比方说数据修改),尤其是一些不可控的批量更新或删除,通常都建议备份后操作.不过不怕万一,就怕一万,有备无患总是好的.在线 ...
- Oracl闪回数据命令。
当数据库操作没有备份,并且误删数据.可闪回任何 当前闪回15分钟前数据库状态. alter table BASE_APPOINT_LOG enable row movement;flashback ...
- [Hibernate] - Select/Update/Delete/Insert
Java bean: package com.my.bean; import java.util.Date; public class WorkPack { private String uWorkP ...
随机推荐
- lucene相关
lucene相关: 应用领域: 互联网全文检索引擎(比如百度, 谷歌, 必应) 站内全文检索引擎(淘宝, 京东搜索功能) 优化数据库查询(因为数据库中使用like关键字是全表扫描也就是顺序扫描算法 ...
- springBoot的事物管理
springBoot的事物管理 1:springBoot 整合单数据源事物: Spring Boot 使用事务非常简单,首先使用注解 @EnableTransactionManagement 开启事务 ...
- 044、vloume声明周期管理（2019-03-07 周四）
参考https://www.cnblogs.com/CloudMan6/p/7214828.html 如果Data Volume 中存放的是重要的应用数据,如何管理volume对应用至关重要. ...
- 编辑距离算法以及它的C#实现
原文:https://www.cnblogs.com/shihuajie/p/5772173.html 注意,原文中有以下表述不当的地方 “扫描两字符串(n*m级的),如果:str1 == str2[ ...
- SQL Server进阶（十一）存储过程
存储过程和函数的区别 存储过程是第一次编译之后就会被存储的下来的预编译对象,之后无论何时调用它都会去执行已经编译好的代码.而函数每次执行都需要编译一次.总结下来有下面几个区别: 基本不同: 函数必须有 ...
- JS监听微信、支付宝等移动app及浏览器的返回、后退、上一页按钮的事件方法
$(function(){ pushHistory(); window.addEventListener("popstate", function(e) { alert(" ...
- 【SRM-07 D】天才麻将少女KPM
Description 天才麻将少女KPM立志要在日麻界闯出一番名堂.KPM上周叒打了n场麻将,但她这次又没控分,而且因为是全市参与的麻将大赛,所以她的名次范围是0..10^5.名次可能等于0是因为K ...
- oracle行转列函数以及一些窗口函数（PIVOT ,OVER）
可以查看博客园上传的文件 pivot 除了要转成列的字段(classnum,classid)值不一样,其他值都应该相同,否则转的数据有问题
- C++ 初始化函数
初始化函数:OnInitDialog() 在这个位置添加初始化代码
- Json转Hashtable的转换
json 转换为Hashtable /// <summary> /// Json转Hashtable /// </summary> /// <param name=&qu ...