Percona 工具包 pt-online-schema-change 简介

mysql的在线表结构修改，因为低效和阻塞读写。一直被诟病。至于ALTER TABLE 的原理，参看我上一篇文章。MySQL在线修改大表结构。看完后，发现的问题是还是会锁的，且对于在线更新的这块也是不能很好的处理，所以只能从理论上来理解，应用到在线库还是有问题的。不能保证数据的最新。

今天看到percona推出的工具包中的online-schema-change 和facebook推出的一样。大概原理是一致的。如何安装使用详看官网文档。http://www.percona.com/doc/percona-toolkit/2.1/pt-online-schema-change.html

注意文档上说的很清楚。需要注意的是对于两个参数的使用。--dry-run 和--execute 的区别使用。建立先使用--dry-run和--print查看更改的流程。OK后执行--execute 。

引用官网说明如下:

   --dry-run

Create and alter the new table, but do not create triggers, copy data, or replace the original table.

  --execute

Indicate that you have read the documentation and want to alter the table. You must specify this option to alter the table. If you do not, then the tool will only perform some safety checks and exit. This helps ensure that you have read the documentation and understand how to use this tool. If you have not read the documentation, then do not specify this option.

如下是自己执行--dry-run运行的过程。因为是在测试库中，没有新的记录出现。但是执行流程还是可以看出来。屏蔽了密码和host。

root@rb-11:/home/hzwuzhimin# pt-online-schema-change --user=root --password=*** --host='****'  --lock-wait-time=120   --alter="add column dd int(11) default 0"    D=wzm,t=tmp20121127_user_text_monitor_config,F=/home/hzwuzhimin/mysql/my.cnf,S=/home/hzwuzhimin/mysql/node/mysqld.sock,P=4333  --dry-run --print

Starting a dry run.  `wzm`.`tmp20121127_user_text_monitor_config` will not be altered.  Specify --execute instead of --dry-run to alter the table.

Creating new table...

CREATE TABLE `wzm`.`_tmp20121127_user_text_monitor_config_new` (

  `id` bigint(20) NOT NULL AUTO_INCREMENT,

  `xml_content` longtext NOT NULL,

  `group_name` varchar(40) NOT NULL DEFAULT '',

  `user_account` varchar(40) NOT NULL DEFAULT '',

  `create_time` bigint(20) NOT NULL,

  `modify_time` bigint(20) NOT NULL,

  `modify_user_account` varchar(40) NOT NULL DEFAULT '',

  `v` bigint(20) NOT NULL DEFAULT '1',

  `aa` int(11) DEFAULT '0',

  `bb` int(11) DEFAULT '0',

  `cc` int(11) DEFAULT '0',

  PRIMARY KEY (`id`)

) ENGINE=MyISAM AUTO_INCREMENT=244 DEFAULT CHARSET=gbk

Created new table wzm._tmp20121127_user_text_monitor_config_new OK.

Altering new table...

ALTER TABLE `wzm`.`_tmp20121127_user_text_monitor_config_new` add column dd int(11) default 0

Altered `wzm`.`_tmp20121127_user_text_monitor_config_new` OK.

Not creating triggers because this is a dry run.

CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_del` AFTER DELETE ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW DELETE IGNORE FROM `wzm`.`_tmp20121127_user_text_monitor_config_new` WHERE `wzm`.`_tmp20121127_user_text_monitor_config_new`.`id` <=> OLD.`id`

CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_upd` AFTER UPDATE ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW REPLACE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) VALUES (NEW.`id`, NEW.`xml_content`, NEW.`group_name`, NEW.`user_account`, NEW.`create_time`, NEW.`modify_time`, NEW.`modify_user_account`, NEW.`v`, NEW.`aa`, NEW.`bb`, NEW.`cc`)

CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_ins` AFTER INSERT ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW REPLACE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) VALUES (NEW.`id`, NEW.`xml_content`, NEW.`group_name`, NEW.`user_account`, NEW.`create_time`, NEW.`modify_time`, NEW.`modify_user_account`, NEW.`v`, NEW.`aa`, NEW.`bb`, NEW.`cc`)

Not copying rows because this is a dry run.

 

INSERT LOW_PRIORITY IGNORE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) SELECT `id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc` FROM `wzm`.`tmp20121127_user_text_monitor_config` LOCK IN SHARE MODE /*pt-online-schema-change 7498 copy table*/

Not swapping tables because this is a dry run.

Not dropping old table because this is a dry run.

Not dropping triggers because this is a dry run.

DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_del`;

DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_upd`;

DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_ins`;

Dropping new table...

DROP TABLE IF EXISTS `wzm`.`_tmp20121127_user_text_monitor_config_new`;

Dropped new table OK.

Dry run complete.  `wzm`.`tmp20121127_user_text_monitor_config` was not altered.

 

执行--execute后的过程如下

root@rb-11:/home/hzwuzhimin# pt-online-schema-change --user=root --password=*** --host='*****'  --lock-wait-time=120   --alter="add column dd int(11) default 0"    D=wzm,t=tmp20121127_user_text_monitor_config,F=/home/hzwuzhimin/mysql/my.cnf,S=/home/hzwuzhimin/mysql/node/mysqld.sock,P=4333 --execute --print

Altering `wzm`.`tmp20121127_user_text_monitor_config`...

Creating new table...

CREATE TABLE `wzm`.`_tmp20121127_user_text_monitor_config_new` (

  `id` bigint(20) NOT NULL AUTO_INCREMENT,

  `xml_content` longtext NOT NULL,

  `group_name` varchar(40) NOT NULL DEFAULT '',

  `user_account` varchar(40) NOT NULL DEFAULT '',

  `create_time` bigint(20) NOT NULL,

  `modify_time` bigint(20) NOT NULL,

  `modify_user_account` varchar(40) NOT NULL DEFAULT '',

  `v` bigint(20) NOT NULL DEFAULT '1',

  `aa` int(11) DEFAULT '0',

  `bb` int(11) DEFAULT '0',

  `cc` int(11) DEFAULT '0',

  PRIMARY KEY (`id`)

) ENGINE=MyISAM AUTO_INCREMENT=244 DEFAULT CHARSET=gbk

Created new table wzm._tmp20121127_user_text_monitor_config_new OK.

Altering new table...

ALTER TABLE `wzm`.`_tmp20121127_user_text_monitor_config_new` add column dd int(11) default 0

Altered `wzm`.`_tmp20121127_user_text_monitor_config_new` OK.

Creating triggers...

CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_del` AFTER DELETE ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW DELETE IGNORE FROM `wzm`.`_tmp20121127_user_text_monitor_config_new` WHERE `wzm`.`_tmp20121127_user_text_monitor_config_new`.`id` <=> OLD.`id`

CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_upd` AFTER UPDATE ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW REPLACE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) VALUES (NEW.`id`, NEW.`xml_content`, NEW.`group_name`, NEW.`user_account`, NEW.`create_time`, NEW.`modify_time`, NEW.`modify_user_account`, NEW.`v`, NEW.`aa`, NEW.`bb`, NEW.`cc`)

CREATE TRIGGER `pt_osc_wzm_tmp20121127_user_text_monitor_config_ins` AFTER INSERT ON `wzm`.`tmp20121127_user_text_monitor_config` FOR EACH ROW REPLACE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) VALUES (NEW.`id`, NEW.`xml_content`, NEW.`group_name`, NEW.`user_account`, NEW.`create_time`, NEW.`modify_time`, NEW.`modify_user_account`, NEW.`v`, NEW.`aa`, NEW.`bb`, NEW.`cc`)

Created triggers OK.

Copying approximately 123 rows...

INSERT LOW_PRIORITY IGNORE INTO `wzm`.`_tmp20121127_user_text_monitor_config_new` (`id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc`) SELECT `id`, `xml_content`, `group_name`, `user_account`, `create_time`, `modify_time`, `modify_user_account`, `v`, `aa`, `bb`, `cc` FROM `wzm`.`tmp20121127_user_text_monitor_config` LOCK IN SHARE MODE /*pt-online-schema-change 8270 copy table*/

Copied rows OK.

Swapping tables...

RENAME TABLE `wzm`.`tmp20121127_user_text_monitor_config` TO `wzm`.`_tmp20121127_user_text_monitor_config_old`, `wzm`.`_tmp20121127_user_text_monitor_config_new` TO `wzm`.`tmp20121127_user_text_monitor_config`

Swapped original and new tables OK.

Dropping old table...

DROP TABLE IF EXISTS `wzm`.`_tmp20121127_user_text_monitor_config_old`

Dropped old table `wzm`.`_tmp20121127_user_text_monitor_config_old` OK.

Dropping triggers...

DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_del`;

DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_upd`;

DROP TRIGGER IF EXISTS `wzm`.`pt_osc_wzm_tmp20121127_user_text_monitor_config_ins`;

Dropped triggers OK.

Successfully altered `wzm`.`tmp20121127_user_text_monitor_config`.

 

总结：

实际上通过print出来的日志。我们就能很清楚的看到整个在线更新的流程。

1：创建跟需要修改的表结构一样的表结构。命名为旧表名_new;

2:   根据alter 条件修改新表结构为需要后的表结果；

3：创建触发器，对于修改表和新表的关联，对于INSERT,UPDATE,DELETE的操作。保证两表记录同步。(因为在线更新是不阻塞用户对修改表读写）;

4：根据触发条件插入记录到新表中，直到复制完成；

5：重命名表，先将修改的表TAB 重名名成TAB_OLD,  TAB_NEW 重名成TAB ;

6:  然后删除TAB_OLD表。成功后，删除触发器。执行完成。