Percona-Tookit工具包之pt-duplicate-key-checker

 

Preface

 

    I suppose that we have a requirement of checking out how many duplicated indexes on a certain table or even on several specific databases.what will you do then?

 

Introduce

 

    pt-duplicate-key-checker is the very tool which can help us to check out the replicated keys and foreign keys in our databases.

 

Procedure

 

Usage

 pt-duplicate-key-checker [OPTIONS] [DSN]

Common parameter

 --clustered //Treats the columns of pripary key to be redundent like secondary key does.(default "yes")
 --key-types //Specify the type of index to check.(default "fk" means both of f&k,f=foreign keys,k=ordinary keys)
 --database //Specify the database you want to check.
 --tables //Specify the tables you want to check.
 --ignore-databases //Specify the ignoring database.
 --ignore-tables //Specify the ignoring tables.
 --ignore-order //Treats the key(a,b) to be duplicated when it is compared with the key(b,a)
 --sql //Generate dropping index sql statment for those duplicated indexes.

Example

 

Check the indexes on a "sbtest4".

 (zlm@192.168.1.101 )[sysbench]>show tables;
 +--------------------+
 | Tables_in_sysbench |
 +--------------------+
 | sbtest1            |
 | sbtest2            |
 | sbtest3            |
 | sbtest4            |
 +--------------------+
  rows in set (0.00 sec)

 (zlm@192.168.1.101 )[sysbench]>show create table sbtest4\G
 *************************** . row ***************************
        Table: sbtest4
 Create Table: CREATE TABLE `sbtest4` (
   `id` int() NOT NULL AUTO_INCREMENT,
   `k` int() NOT NULL DEFAULT '',
   `c` char() NOT NULL DEFAULT '',
   `pad` char() NOT NULL DEFAULT '',
   PRIMARY KEY (`id`),
   KEY `k_4` (`k`)
 ) ENGINE=InnoDB AUTO_INCREMENT= DEFAULT CHARSET=utf8
  row in set (0.00 sec)

 (zlm@192.168.1.101 )[sysbench]>show index from sbtest4;
 +---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 | Table   | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
 +---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 | sbtest4 |           | PRIMARY  |             | id          | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | k_4      |             | k           | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 +---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
  rows in set (0.00 sec)

 (zlm@192.168.1.101 )[sysbench]>

Add an index on column id and k in table "sbtest4".

 (zlm@192.168.1.101 )[sysbench]>alter table sbtest4 add index idx_id_k1(id,k);
 Query OK,  rows affected (0.11 sec)
 Records:   Duplicates:   Warnings: 

 (zlm@192.168.1.101 )[sysbench]>show index from sbtest4;
 +---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 | Table   | Non_unique | Key_name  | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
 +---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 | sbtest4 |           | PRIMARY   |             | id          | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | k_4       |             | k           | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | idx_id_k1 |             | id          | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | idx_id_k1 |             | k           | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 +---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

Check whether there's a duplicated index in table "sbtest4".

 [root@zlm2 :: ~]
 #pt-duplicate-key-checker -d=sysbench -t=sbtest4
 # ########################################################################
 # Summary of indexes
 # ########################################################################

 # Total Indexes   //No duplicated indexes were found.

 [root@zlm2 :: ~]
 #pt-duplicate-key-checker -dsysbench -tsbtest4
 # ########################################################################
 # Summary of indexes
 # ########################################################################

 # Total Indexes  

Check the conclusion with option "--no-clustered".

 [root@zlm2 :: ~]
 #pt-duplicate-key-checker -dsysbench -tsbtest4 -h192.168.1. -P3306 -uzlm -pzlmzlm --no-clustered
 # ########################################################################
 # sysbench.sbtest4
 # ########################################################################

 # k_4 is a left-prefix of idx_id_k2
 # Key definitions:
 #   KEY `k_4` (`k`),
 #   KEY `idx_id_k2` (`k`,`id`) //The index "idx_id_k2" contains the column in index "k_4" and in the same order.Therefore,"k_4" is indicated redundant.
 # Column types:
 #      `k` int() not null default ''
 #      `id` int() not null auto_increment
 # To remove this duplicate index, execute:
 ALTER TABLE `sysbench`.`sbtest4` DROP INDEX `k_4`;

 # ########################################################################
 # Summary of indexes
 # ########################################################################

 # Size Duplicate Indexes
 # Total Duplicate Indexes
 # Total Indexes            

Check the conclusion with option "--clustered".(Its default is "true" what means we can also omit it)

 [root@zlm2 :: ~]
 #pt-duplicate-key-checker -dsysbench -tsbtest4 -h192.168.1. -P3306 -uzlm -pzlmzlm
 # ########################################################################
 # sysbench.sbtest4
 # ########################################################################

 # k_4 is a left-prefix of idx_id_k2
 # Key definitions:
 #   KEY `k_4` (`k`),
 #   KEY `idx_id_k2` (`k`,`id`)
 # Column types:
 #      `k` int() not null default ''
 #      `id` int() not null auto_increment
 # To remove this duplicate index, execute:
 ALTER TABLE `sysbench`.`sbtest4` DROP INDEX `k_4`;

 # Key idx_id_k2 ends with a prefix of the clustered index
 # Key definitions:
 #   KEY `idx_id_k2` (`k`,`id`)
 #   PRIMARY KEY (`id`),
 # Column types:
 #      `k` int() not null default ''
 #      `id` int() not null auto_increment
 # To shorten this duplicate clustered index, execute:
 ALTER TABLE `sysbench`.`sbtest4` DROP INDEX `idx_id_k2`, ADD INDEX `idx_id_k2` (`k`); //The tool suggests to change the union index to a single column key.

 # ########################################################################
 # Summary of indexes
 # ########################################################################

 # Size Duplicate Indexes
 # Total Duplicate Indexes
 # Total Indexes            

Add another index on column id and k in table "sbtest4".(reverse the order this time)

 (zlm@192.168.1.101 )[sysbench]>alter table sbtest4 add index idx_id_k2(k,id);
 Query OK,  rows affected (0.04 sec)
 Records:   Duplicates:   Warnings: 

 (zlm@192.168.1.101 )[sysbench]>show index from sbtest4;
 +---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 | Table   | Non_unique | Key_name  | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
 +---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
 | sbtest4 |           | PRIMARY   |             | id          | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | k_4       |             | k           | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | idx_id_k1 |             | id          | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | idx_id_k1 |             | k           | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | idx_id_k2 |             | k           | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 | sbtest4 |           | idx_id_k2 |             | id          | A         |         |     NULL | NULL   |      | BTREE      |         |               |
 +---------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
  rows in set (0.00 sec)

Check the conclusion with default option again.

 [root@zlm2 :: ~]
 #pt-duplicate-key-checker -dsysbench -tsbtest4 -h192.168.1. -P3306 -uzlm -pzlmzlm
 # ########################################################################
 # sysbench.sbtest4
 # ########################################################################

 # k_4 is a left-prefix of idx_id_k2
 # Key definitions:
 #   KEY `k_4` (`k`),
 #   KEY `idx_id_k2` (`k`,`id`)
 # Column types:
 #      `k` int() not null default ''
 #      `id` int() not null auto_increment
 # To remove this duplicate index, execute:
 ALTER TABLE `sysbench`.`sbtest4` DROP INDEX `k_4`;

 # Key idx_id_k2 ends with a prefix of the clustered index
 # Key definitions:
 #   KEY `idx_id_k2` (`k`,`id`)
 #   PRIMARY KEY (`id`),
 # Column types:
 #      `k` int() not null default ''
 #      `id` int() not null auto_increment
 # To shorten this duplicate clustered index, execute:
 ALTER TABLE `sysbench`.`sbtest4` DROP INDEX `idx_id_k2`, ADD INDEX `idx_id_k2` (`k`);

 # ########################################################################
 # Summary of indexes
 # ########################################################################

 # Size Duplicate Indexes
 # Total Duplicate Indexes
 # Total Indexes            

 Check the conclusion another time with option "--ignore-order".

 [root@zlm2 :: ~]
 #pt-duplicate-key-checker --databases=sysbench --tables=sbtest4 --ignore-order -h192.168.1. -P3306 -uzlm -pzlmzlm
 # ########################################################################
 # sysbench.sbtest4
 # ########################################################################

 # idx_id_k2 is a duplicate of idx_id_k1
 # Key definitions:
 #   KEY `idx_id_k2` (`k`,`id`)
 #   KEY `idx_id_k1` (`id`,`k`),
 # Column types:
 #      `k` int() not null default ''
 #      `id` int() not null auto_increment
 # To remove this duplicate index, execute:
 ALTER TABLE `sysbench`.`sbtest4` DROP INDEX `idx_id_k2`; //Why no dropping "idx_id_k1"?Beacause "idx_id_k2" is also a duplicate of "k_4" as we can see above.

 # ########################################################################
 # Summary of indexes
 # ########################################################################

 # Size Duplicate Indexes
 # Total Duplicate Indexes
 # Total Indexes            

Summary

• Notice the difference between long option and short option,do not mix them up.
• pt-duplicate-key-checker is quit convenient to generate a summay report of rudundant indexes in specific databases.
• We can execute it with script in a certain interval of time and collect the information in a flat logfile for future analysis.
• It also provides us the pertinent SQL statement to adjust the indexes in target tables.
• Further more,we'd better compare the result of the tool with the data in sys.schema_redundant_indexes(5.7 or above).