mysql的在线表结构修改,因为低效和阻塞读写。一直被诟病。至于ALTER TABLE 的原理,参看我上一篇文章。MySQL在线修改大表结构。看完后,发现的问题是还是会锁的,且对于在线更新的这块也是不能很好的处理,所以只能从理论上来理解,应用到在线库还是有问题的。不能保证数据的最新。

今天看到percona推出的工具包中的online-schema-change 和facebook推出的一样。大概原理是一致的。如何安装使用详看官网文档。http://www.percona.com/doc/percona-toolkit/2.1/pt-online-schema-change.html
注意文档上说的很清楚。需要注意的是对于两个参数的使用。--dry-run 和--execute 的区别使用。建立先使用--dry-run和--print查看更改的流程。OK后执行--execute 。
引用官网说明如下:
   --dry-run

Create and alter the new table, but do not create triggers, copy data, or replace the original table.

  --execute

Indicate that you have read the documentation and want to alter the table. You must specify this option to alter the table. If you do not, then the tool will only perform some safety checks and exit. This helps ensure that you have read the documentation and understand how to use this tool. If you have not read the documentation, then do not specify this option.

如下是自己执行--dry-run运行的过程。因为是在测试库中,没有新的记录出现。但是执行流程还是可以看出来。屏蔽了密码和host。

root@rb-11:/home/hzwuzhimin# pt-online-schema-change --user=root --password=*** --host='****'  --lock-wait-time=120   --alter="add column dd int(11) default 0"    D=wzm,t=tmp20121127_user_text_monitor_config,F=/home/hzwuzhimin/mysql/my.cnf,S=/home/hzwuzhimin/mysql/node/mysqld.sock,P=4333  --dry-run --print
Starting a dry run.  `wzm`.`tmp20121127_user_text_monitor_config` will not be altered.  Specify --execute instead of --dry-run to alter the table.
Creating new table...
CREATE TABLE `wzm`.`_tmp20121127_user_text_monitor_config_new` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `xml_content` longtext NOT NULL,
  `group_name` varchar(40) NOT NULL DEFAULT '',
  `user_account` varchar(40) NOT NULL DEFAULT '',
  `create_time` bigint(20) NOT NULL,
  `modify_time` bigint(20) NOT NULL,
  `modify_user_account` varchar(40) NOT NULL DEFAULT '',
  `v` bigint(20) NOT NULL DEFAULT '1',
  `aa` int(11) DEFAULT '0',
  `bb` int(11) DEFAULT '0',
  `cc` int(11) DEFAULT '0',
  PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=244 DEFAULT CHARSET=gbk
Created new table wzm._tmp20121127_user_text_monitor_config_new OK.
Altering new table...
ALTER TABLE `wzm`.`_tmp20121127_user_text_monitor_config_new` add column dd int(11) default 0
Altered `wzm`.`_tmp20121127_user_text_monitor_config_new` OK.
Not creating triggers because this is a dry run.
CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_del` AFTER DELETE ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW DELETE IGNORE FROM `wzm`.`_tmp20121127_user_text_monitor_config_new` WHERE `wzm`.`_tmp20121127_user_text_monitor_config_new`.`id` <=> OLD.`id`
CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_upd` AFTER UPDATE ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW REPLACE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) VALUES (NEW.`id`, NEW.`xml_content`, NEW.`group_name`, NEW.`user_account`, NEW.`create_time`, NEW.`modify_time`, NEW.`modify_user_account`, NEW.`v`, NEW.`aa`, NEW.`bb`, NEW.`cc`)
CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_ins` AFTER INSERT ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW REPLACE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) VALUES (NEW.`id`, NEW.`xml_content`, NEW.`group_name`, NEW.`user_account`, NEW.`create_time`, NEW.`modify_time`, NEW.`modify_user_account`, NEW.`v`, NEW.`aa`, NEW.`bb`, NEW.`cc`)
Not copying rows because this is a dry run.
 
INSERT LOW_PRIORITY IGNORE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) SELECT `id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc` FROM `wzm`.`tmp20121127_user_text_monitor_config` LOCK IN SHARE MODE /*pt-online-schema-change 7498 copy table*/
Not swapping tables because this is a dry run.
Not dropping old table because this is a dry run.
Not dropping triggers because this is a dry run.
DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_del`;
DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_upd`;
DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_ins`;
Dropping new table...
DROP TABLE IF EXISTS `wzm`.`_tmp20121127_user_text_monitor_config_new`;
Dropped new table OK.
Dry run complete.  `wzm`.`tmp20121127_user_text_monitor_config` was not altered.
 
执行--execute后的过程如下
root@rb-11:/home/hzwuzhimin# pt-online-schema-change --user=root --password=*** --host='*****'  --lock-wait-time=120   --alter="add column dd int(11) default 0"    D=wzm,t=tmp20121127_user_text_monitor_config,F=/home/hzwuzhimin/mysql/my.cnf,S=/home/hzwuzhimin/mysql/node/mysqld.sock,P=4333 --execute --print
Altering `wzm`.`tmp20121127_user_text_monitor_config`...
Creating new table...
CREATE TABLE `wzm`.`_tmp20121127_user_text_monitor_config_new` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `xml_content` longtext NOT NULL,
  `group_name` varchar(40) NOT NULL DEFAULT '',
  `user_account` varchar(40) NOT NULL DEFAULT '',
  `create_time` bigint(20) NOT NULL,
  `modify_time` bigint(20) NOT NULL,
  `modify_user_account` varchar(40) NOT NULL DEFAULT '',
  `v` bigint(20) NOT NULL DEFAULT '1',
  `aa` int(11) DEFAULT '0',
  `bb` int(11) DEFAULT '0',
  `cc` int(11) DEFAULT '0',
  PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=244 DEFAULT CHARSET=gbk
Created new table wzm._tmp20121127_user_text_monitor_config_new OK.
Altering new table...
ALTER TABLE `wzm`.`_tmp20121127_user_text_monitor_config_new` add column dd int(11) default 0
Altered `wzm`.`_tmp20121127_user_text_monitor_config_new` OK.
Creating triggers...
CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_del` AFTER DELETE ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW DELETE IGNORE FROM `wzm`.`_tmp20121127_user_text_monitor_config_new` WHERE `wzm`.`_tmp20121127_user_text_monitor_config_new`.`id` <=> OLD.`id`
CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_upd` AFTER UPDATE ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW REPLACE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) VALUES (NEW.`id`, NEW.`xml_content`, NEW.`group_name`, NEW.`user_account`, NEW.`create_time`, NEW.`modify_time`, NEW.`modify_user_account`, NEW.`v`, NEW.`aa`, NEW.`bb`, NEW.`cc`)
CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_ins` AFTER INSERT ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW REPLACE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) VALUES (NEW.`id`, NEW.`xml_content`, NEW.`group_name`, NEW.`user_account`, NEW.`create_time`, NEW.`modify_time`, NEW.`modify_user_account`, NEW.`v`, NEW.`aa`, NEW.`bb`, NEW.`cc`)
Created triggers OK.
Copying approximately 123 rows...
INSERT LOW_PRIORITY IGNORE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) SELECT `id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc` FROM `wzm`.`tmp20121127_user_text_monitor_config` LOCK IN SHARE MODE /*pt-online-schema-change 8270 copy table*/
Copied rows OK.
Swapping tables...
RENAME TABLE `wzm`.`tmp20121127_user_text_monitor_config` TO `wzm`.`_tmp20121127_user_text_monitor_config_old`, `wzm`.`_tmp20121127_user_text_monitor_config_new` TO `wzm`.`tmp20121127_user_text_monitor_config`
Swapped original and new tables OK.
Dropping old table...
DROP TABLE IF EXISTS `wzm`.`_tmp20121127_user_text_monitor_config_old`
Dropped old table `wzm`.`_tmp20121127_user_text_monitor_config_old` OK.
Dropping triggers...
DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_del`;
DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_upd`;
DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_ins`;
Dropped triggers OK.
Successfully altered `wzm`.`tmp20121127_user_text_monitor_config`.
 
总结:
实际上通过print出来的日志。我们就能很清楚的看到整个在线更新的流程。
1:创建跟需要修改的表结构一样的表结构。命名为旧表名_new;
2:   根据alter 条件修改新表结构为需要后的表结果;
3:创建触发器,对于修改表和新表的关联,对于INSERT,UPDATE,DELETE的操作。保证两表记录同步。(因为在线更新是不阻塞用户对修改表读写);
4:根据触发条件插入记录到新表中,直到复制完成;
5:重命名表,先将修改的表TAB 重名名成TAB_OLD,  TAB_NEW 重名成TAB ;
6:  然后删除TAB_OLD表。成功后,删除触发器。执行完成。
 

Percona 工具包 pt-online-schema-change 简介的更多相关文章

  1. schema change + ogg 变更手册

    Check OGG  until no data queuing in replication process:testRO:a)login  test5 –l oggmgrb)oggc)#ggsci ...

  2. Online Schema Change for MySQL

    It is great to be able to build small utilities on top of an excellent RDBMS. Thank you MySQL. This ...

  3. AppBoxFuture(四). 随需而变-Online Schema Change

      需求变更是信息化过程中的家常便饭,而在变更过程中如何尽可能小的影响在线业务是比较头疼的事情.举个车联网监控的例子:原终端设备上传车辆的经纬度数据,新的终端设备支持同时上传速度数据,而旧的车辆状态表 ...

  4. 安装percona工具包

    1.安装percona源 sudo yum install http://www.percona.com/downloads/percona-release/redhat/0.1-4/percona- ...

  5. 数据库schema的简介

    [参考]自百度百科 数据库中的Schema,为数据库对象的集合,一个用户一般对应一个schema. 官方定义如下: A schema is a collection of database objec ...

  6. Oracle数据库Schema的简介

    百度文库中 Schema 的解释: 数据库中的Schema,为数据库对象的集合,一个用户一般对应一个schema. 官方定义如下: A schema is a collection of databa ...

  7. Online, Asynchronous Schema Change in F1

    F1: A Distributed SQL Database That Scales   http://disksing.com/understanding-f1-schema-change   ma ...

  8. Online Schema Upgrade in MySQL Galera Cluster using TOI Method

    http://severalnines.com/blog/online-schema-upgrade-mysql-galera-cluster-using-toi-method     As a fo ...

  9. Mysql 主从一致校验工具------Maatkit工具包

    Maatkit工具包 http://www.maatkit.org/ 简介 maatkit是一个开源的工具包,为mysql日常管理提供了帮助.目前,已被Percona公司收购并维护.其中: mk-ta ...

随机推荐

  1. 转载: Flex 布局教程

    demo:页面二等分 .flex-box { display: -webkit-flex; /* Safari */ display: flex; flex-direction: row; justi ...

  2. jQuery 对象访问 index([selector|element])

    搜索匹配的元素,并返回相应元素的索引值,从0开始计数. 如果不给 .index() 方法传递参数,那么返回值就是这个jQuery对象集合中第一个元素相对于其同辈元素的位置. 如果参数是一组DOM元素或 ...

  3. linux禁用触摸板驱动

    Method 1: 终端输入如下命令: sudo modprobe -r psmouse 如果打开触摸板就是: sudo modprobe psmouse ------- Method 2: 第一步: ...

  4. Direct Line Guidance Odometry论文阅读笔记

    摘要: 本文特色:使用线引导关键点的选择.本文提出这个的论点是:线上的点比图像的其他部分的点更好,而且线上存在更好的关键点.选择线上的点可以筛选过滤掉不太明显的点,从而提高效率. 点和线: 系统使用点 ...

  5. sqli_labs第一关

    安装 从https://github.com/Audi-1/sqli-labs下载源代码 搭建环境用的是phpstudy 编辑sqli\sql-connections\db-creds.inc文件 修 ...

  6. Spoken English Practice(I'm gonna do something I never thought I'd be able to)

    绿色:连读:                  红色:略读:               蓝色:浊化:               橙色:弱读     下划线_为浊化 口语蜕变(2017/7/6) 英 ...

  7. Python菜鸟之路:Django 数据验证之钩子和Form表单验证

    一.钩子功能提供的数据验证 对于数据验证,django会执行 full_clean()方法进行验证.full_clean验证会经历几个步骤,首先,对于model的每个字段进行正则验证,正则验证通过后, ...

  8. Python菜鸟之路:Django 文件上传的几种方式

    方式一:通过form表单中,html input 标签的“file”完成 # 前端代码uoload.html <form method="post" action=" ...

  9. Qt 使用 lambda 表达式做为槽函数时为什么使用 QObject::sender() 获取到的发送信号对象指针为空?

    /*! Returns a pointer to the object that sent the signal, if called in a slot activated by a signal; ...

  10. 原!操作 excel 03/07

    参考 所用jar包: poi-3.11.jar poi-ooxml-3.11.jar poi-ooxml-schemas-3.11.jar /* * Project: fusion-may-open- ...