MySQL Test Suite使用

MySQL Test Suite使用

MySQL自动测试套件（The MySQL Test Suite）用于对MySQL程序进行测试，包括各种功能与存储引擎。包含于MySQL与MariaDB版本代码中，位于mysql-test目录下。总体测试脚本为mysql-test-run.pl。该perl脚本通过调用各种已有的测试脚本，将测试结果与预置的result文件做对比来判断测试是否通过。（任何一点差异都会导致测试失败，包括预期之外的warning）

 

 

一、主要目录介绍

• • include

包含.inc文件，用于测试开始时判断是否满足测试条件，在测试用通过source命令引入

如hava_innodb.inc:

 

1 2 3 4

if (`SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.ENGINES WHERE engine = 'innodb' AND support IN ('YES', 'DEFAULT', 'ENABLED')`) {              --skip Test requires InnoDB. }

如不满足，则会跳过需要该验证的测试。

• • suite

包含所有测试suite，每一个suite为一个测试用例集，具体用例集在2节介绍

 

 

• • t

1、测试文件

例：1st.test

 

1 2 3 4 5

# # Check that we haven't any strange new tables or databases # show databases; show tables in mysql;

上面内容即为1st这个测试用例的测试脚本

 2、配置文件

例：innodb_bug53674-master.opt

–loose-innodb-locks-unsafe-for-binlog –binlog-format=mixed

为该测试需要的特殊配置项

 

3、脚本文件，在测试开始之前执行的脚本

例：rpl_misc_functions-slave.sh

rm -f $MYSQLTEST_VARDIR/master-data/test/rpl_misc_functions.outfile

 

 

• • r

结果文件，判断是否通过的依据

使用上面的例子：1st.result

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

show databases; Database information_schema mtr mysql performance_schema test show tables in mysql; Tables_in_mysql column_stats columns_priv db event func general_log gtid_slave_pos help_category help_keyword help_relation help_topic host index_stats innodb_index_stats innodb_table_stats plugin proc procs_priv proxies_priv roles_mapping servers slow_log table_stats tables_priv time_zone time_zone_leap_second time_zone_name time_zone_transition time_zone_transition_type user

执行完1st用例会严格比对该文件。完全一致则测试通过。

• • std_date

有些测试需要用到一些标准数据。

 

• • var

测试默认使用该文件夹下的my.cnf文件

 

 

二、测试用例集

./mysql-test-run.pl不带任何参数表示执行所有测试用例集。使用—suite=suitename来指定单独执行suitename目录下的所有测试用例。

测试用例集包括：

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

main：主要测试用例 archive binlog csv encryption：加密功能 engines federated funcs_1：额外功能（包括视图、存储过程、INFORMATION_SCHEMA等） funcs_2：额外功能（字符集等） galera galera_3nodes handler heap innodb innodb_fts：innodb全文索引 innodb_zip：innodb压缩 jp：日本字符相关测试 large_tests： maria mtr mtr2 multi_source ndb optimizer_unfixed_bugs parts percona perfschema perfschema_stress plugins roles rpl storage_engine stress sys_vars unit vcol wsrep

三、主要参数

force：默认情况下，只要遇到一个用例出错，测试程序就会退出，在指定force的情况下，测试程序会继续执行下面的测试（但是最多发现10个错误还是会退出）

suite：指定使用的测试suite

do-test：会以输入的字符串进行匹配用例执行

skip-test：会以输入的字符串进行匹配跳过用例执行

big-test：执行标记为big的测试用例。因为用例较大、耗时较长，标记为big的用例默认不会执行。输入两遍big-test则只执行标记为big的测试用例

recored：重新生成结果文件

mysqld：传递启动参数给mysqld，每个参数需要分别指定一个mysqld

其他还有很多选项，可以使用./mtr –help查看

 

如果需要在服务启动前执行一些脚本，可以写在 t/testname.sh文件中，由mtr自动执行。

 

 

四、测试举例

例1：

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

[lzk@redhat64-26 mysql-test]$ ./mtr  --force --big-test --suite=archive Logging: ./mtr  --force --big-test --suite=archive vardir: /home/lzk/mysql-test/var Checking leftover processes... Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB  - SSL connections supported Using suites: archive Collecting tests... Installing system database...   ==============================================================================   TEST                                      RESULT   TIME (ms) or COMMENT --------------------------------------------------------------------------   worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 archive.archive_debug                    [ skipped ]  Requires debug build archive.partition_archive                [ pass ]     74 archive.archive_no_symlink               [ pass ]      5 archive.archive-big                      [ pass ]  298619 archive.archive                          [ pass ]    417 archive.archive_bitfield                 [ pass ]     37 archive.archive_gis                      [ pass ]     66 archive.archive_symlink                  [ pass ]     11 archive.discover                         [ pass ]     47 archive.discover_5438                    [ pass ]      7 archive.mysqlhotcopy_archive             [ pass ]   1035 archive.repair                           [ pass ]      3 archive.archive_plugin                   [ pass ]      6 -------------------------------------------------------------------------- The servers were restarted 3 times Spent 300.327 of 309 seconds executing testcases   Completed: All 12 tests were successful.   1 tests were skipped, 0 by the test itself.

 

可以看出共完成12个测试用例，全部通过。1个用例必须在debug模式下测试，所以跳过。测试过程中服务重启3次。测试默认使用16000-16019端口。

 

例2：

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

[lzk@redhat64-26 mysql-test]$  ./mtr binlog_innodb_row Logging: ./mtr  binlog_innodb_row vardir: /home/lzk/mysql-test/var Checking leftover processes...  - found old pid 28610 in 'mysqld.1.pid', killing it...    process did not exist! Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB  - SSL connections supported klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_550) Collecting tests... Installing system database...   ==============================================================================   TEST                                      RESULT   TIME (ms) or COMMENT --------------------------------------------------------------------------   worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 binlog.binlog_innodb_row 'innodb_plugin,row' [ pass ]   3361 binlog.binlog_innodb_row 'row,xtradb'    [ pass ]   3349 -------------------------------------------------------------------------- The servers were restarted 1 times Spent 6.710 of 17 seconds executing testcases   Completed: All 2 tests were successful.

每个测试结束前，mtr会检索error日志，如果发现warning或error，则测试失败。

再来看两个失败用例：

（截取部分日志）

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

innodb.innodb_bug53674 'innodb_plugin'   [ fail ]  Found warnings/errors in server log file!         Test ended at 2016-03-24 20:57:36 line 2016-03-24 20:57:35 7fd0fc6fc7e0 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. ^ Found warnings in /home/lzk/mysql-test/var/log/mysqld.1.err ok    - saving '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-innodb_plugin/' to '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-innodb_plugin/' innodb.innodb_bug53674 'xtradb'          [ fail ]  Found warnings/errors in server log file!         Test ended at 2016-03-24 20:57:37 line 2016-03-24 20:57:36 7f274c58b7e0 InnoDB: Warning: Using innodb_locks_unsafe_for_binlog is DEPRECATED. This option may be removed in future releases. Please use READ COMMITTED transaction isolation level instead, see http://dev.mysql.com/doc/refman/5.6/en/set-transaction.html. ^ Found warnings in /home/lzk/mysql-test/var/log/mysqld.1.err ok    - saving '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-xtradb/' to '/home/lzk/mysql-test/var/log/innodb.innodb_bug53674-xtradb/'

分析日志可以看出是在测试过程中在错误日志中发现了warnings/errors。原因是使用了废弃的参数innodb_locks_unsafe_for_binlog导致warning。

找到并修改配置文件：innodb_bug53674-master.opt

将–loose-innodb-locks-unsafe-for-binlog –binlog-format=mixed

修改为：–transaction-isolation=READ-COMMITTED –binlog-format=mixed

 

重新测试：

 

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

[lzk@redhat64-26 mysql-test]$ ./mtr innodb_bug53674 Logging: ./mtr  innodb_bug53674 vardir: /home/lzk/mysql-test/var Checking leftover processes... Removing old var directory... Creating var directory '/home/lzk/mysql-test/var'... Checking supported features... MariaDB Version 10.1.12-MariaDB  - SSL connections supported klist: No credentials cache found (ticket cache FILE:/tmp/krb5cc_550) Collecting tests... Installing system database...   ==============================================================================   TEST                                      RESULT   TIME (ms) or COMMENT --------------------------------------------------------------------------   worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019 innodb.innodb_bug53674 'innodb_plugin'   [ pass ]    211 innodb.innodb_bug53674 'xtradb'          [ pass ]    205 -------------------------------------------------------------------------- The servers were restarted 1 times Spent 0.416 of 10 seconds executing testcases   Completed: All 2 tests were successful.

不再有warning，测试通过。