基于Avocado 的 QData MySQL自动化测试.md

qdata-mysql 自动化测试概要设计

---

```
│   ├── 1. 依赖环境
│   │   ├
│   │   └─────
│   ├── 2. 配置文件
│   │   ├
│   │   └─────
│   ├
│   ├── 3. 测试流程
│   │   ├
│   │   └─────
│   ├
│   ├── 4. 输出格式
│   │   ├
│   │   └─────
│   │ 
│   ├── 5. 命令行参数
│   │   ├
│   │   └─────
```
#### 依赖环境

• python2.7.9

• avocado-framework (52.0)

• ssh

• avocado需要插件：

avocado-framework-plugin-result-html (52.0)

avocado-framework-plugin-runner-remote (52.0)

avocado-framework-plugin-varianter-yaml-to-mux (52.0)

```

• 前提条件

  • qdata_mysql 版本 1.2.2
  • 集群正常
  • sysbench 压测工具
  • ntp 服务正常（ntp.conf 配置正常 pool.ntp.org）
  • ssh 长连接
  • master 配置数据库sbtest的权限
  • 配置文件 qdata.conf, qdata_for_mysql.conf, qdata_conf_default, 需要备份一份，在恢复集群时，恢复配置文件，再gen_conf

  grant all on sbtest.* to sbtest identified by 'sbtest';
  flush privileges;
  

配置文件

• 用户配置

  • yaml参数配置文件，配置集群master_ip

  #cat data/cluster1.yaml
  master_ip: '10.10.30.101'
  

测试流程

• 集群测试

  • 支持多套集群测试
  • 在data目录下新建参数配置文件，配置每套集群的master_ip,配置方法如下：

  配置文件名称依照cluster1，cluster2，cluster3.....等以此类推
  参数配置集群的master_ip即可，例如master_ip: '10.10.30.101'
  

  指定第一套配置文件执行集群测试后，可新开窗口，再次指定其他配置文件进行测试

  • 通过指定配置文件来进行集群测试：
    
    avocado run --mux-yaml data/cluster1.yaml -- cls.py ha.py lb.py db.py
  • 若测试中断，可以从中断的案例开始执行（指定中断任务的job id）
    
    avocado run --replay 56b4aab --replay-test-status INTERRUPTED
  • 支持只执行测试失败的案例,会跳过执行成功的案例（指定刚执行过的任务job id）
    
    avocado run --replay 56b4aab --replay-test-status FAIL
  • 测试cancel的案例
    
    avocado run --replay 56b4aab --replay-test-status CANCEL

• 屏幕输出

	#avocado run --mux-yaml data/cluster2.yaml -- ha.py db.py lb.py
JOB ID     : eec9fe46c4aee730ee0e4433967244d7781e5c8e
JOB LOG    : /root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/job.log
 (1/9) ha.py:HATest.test_qmcls_start;run-7599: PASS (0.00 s)
 (2/9) ha.py:HATest.test_qmcls_stop;run-7599: FAIL (0.04 s)
 (3/9) ha.py:HATest.test_qmcls_show;run-7599: PASS (0.00 s)
 (4/9) db.py:DBTest.test_db_stop;run-7599: PASS (0.00 s)
 (5/9) db.py:DBTest.test_db_start;run-7599: FAIL (0.04 s)
 (6/9) db.py:DBTest.test_db_backup;run-7599: PASS (0.00 s)
 (7/9) lb.py:LBTest.test_lb_join;run-7599: PASS (0.00 s)
 (8/9) lb.py:LBTest.test_lb_kickout;run-7599: PASS (15.02 s)
 (9/9) lb.py:LBTest.test_set_kickout_time;run-7599: PASS (0.00 s)
RESULTS    : PASS 7 | ERROR 0 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 16.65 s
JOB HTML   : /root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/results.html

• 定位FAIL案例的具体失败信息
  • 打开/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/results.json，根据id，2和5是FAIL，找到对应信息如下：

	    {
            "end": 1501139986.080578,
            "fail_reason": "qmcls stop failed in 10.10.30.102",
            "id": "2-ha.py:HATest.test_qmcls_stop;run-7599",
            "logdir": "/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/test-results/2-ha.py:HATest.test_qmcls_stop;run-7599",
            "logfile": "/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/test-results/2-ha.py:HATest.test_qmcls_stop;run-7599/debug.log",
            "start": 1501139986.039281,
            "status": "FAIL",
            "test": "2-ha.py:HATest.test_qmcls_stop;run-7599",
            "time": 0.041297197341918945,
            "url": "2-ha.py:HATest.test_qmcls_stop;run-7599",
            "whiteboard": ""
        },

       {
            "end": 1501139986.487331,
            "fail_reason": "qmcls db start all failed 10.10.30.102",
            "id": "5-db.py:DBTest.test_db_start;run-7599",
            "logdir": "/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/test-results/5-db.py:DBTest.test_db_start;run-7599",
            "logfile": "/root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/test-results/5-db.py:DBTest.test_db_start;run-7599/debug.log",
            "start": 1501139986.447326,
            "status": "FAIL",
            "test": "5-db.py:DBTest.test_db_start;run-7599",
            "time": 0.04000496864318848,
            "url": "5-db.py:DBTest.test_db_start;run-7599",
            "whiteboard": ""
        },

输出格式

• 屏幕打印格式

		#avocado run --mux-yaml data/cluster2.yaml -- ha.py db.py lb.py
JOB ID     : eec9fe46c4aee730ee0e4433967244d7781e5c8e
JOB LOG    : /root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/job.log
 (1/9) ha.py:HATest.test_qmcls_start;run-7599: PASS (0.00 s)
 (2/9) ha.py:HATest.test_qmcls_stop;run-7599: FAIL (0.04 s)
 (3/9) ha.py:HATest.test_qmcls_show;run-7599: PASS (0.00 s)
 (4/9) db.py:DBTest.test_db_stop;run-7599: PASS (0.00 s)
 (5/9) db.py:DBTest.test_db_start;run-7599: FAIL (0.04 s)
 (6/9) db.py:DBTest.test_db_backup;run-7599: PASS (0.00 s)
 (7/9) lb.py:LBTest.test_lb_join;run-7599: PASS (0.00 s)
 (8/9) lb.py:LBTest.test_lb_kickout;run-7599: PASS (15.02 s)
 (9/9) lb.py:LBTest.test_set_kickout_time;run-7599: PASS (0.00 s)
RESULTS    : PASS 7 | ERROR 0 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 16.65 s
JOB HTML   : /root/avocado/job-results/job-2017-07-27T15.19-eec9fe4/results.html

• 结果导出文件格式：

  • 查看详细测试结果:
    
    /root/avocado/job-results/job-2017-07-25T09.14-56b4aab/results.json
    
    
  • 查看日志信息：
    
    /root/avocado/job-results/job-2017-07-25T09.14-56b4aab/job.log
    
    
  • HTML格式结果：
    
    /root/avocado/job-results/job-2017-07-26T14.49-020f341/results.html
    
    

命令行参数

• avocado run／list / variants / config

  • run 执行测试
  • list 列出函数包含的测试案例
  • variants 查看参数配置信息
  • config 查看配置路径信息

• avocado run --mux-yaml

  • --mux-yaml 指定参数配置文件

• 执行命令可指定测试案例函数，即可执行测试案例，例如：
  
  avocado run --mux-yaml data/cluster1.yaml -- ha.py lb.py db.py 或
  
  avocado run ha.py lb.py db.py --mux-yaml data/cluster1.yaml

• 查看配置文件参数详细信息
  
  avocado variants -m --mux-yaml data/cluster1.yaml --summary 2 --variants 2

• 查看日志和结果文件位置
  
  avocado config --datadir

  #avocado config --datadir
  Config files read (in order):
  /usr/local/pyenv/product/etc/avocado/avocado.conf
  /usr/local/pyenv/product/etc/avocado/conf.d/gdb.conf
  /root/.config/avocado/avocado.conf
  
  Avocado replaces config dirs that can't be accessed
  with sensible defaults. Please edit your local config
  file to customize values
  
  Avocado Data Directories:
  base     /var/lib/avocado
  tests    /usr/share/avocado/tests
  data     /var/lib/avocado/data
  logs     /root/avocado/job-results
  

• avocado list nvmtest.py 列出具体的测试用例函数有哪些
  
  avocado list -V/--verbose nvmtest.py 查看详细信息
  
  or avocado list nvmtest.py -V/--verbose

  #avocado list ha.py --verbose
  Type         Test                          Tag(s)
  INSTRUMENTED ha.py:HATest.test_qmcls_start
  INSTRUMENTED ha.py:HATest.test_qmcls_stop
  INSTRUMENTED ha.py:HATest.test_qmcls_show
  
  TEST TYPES SUMMARY
  ==================
  ACCESS_DENIED: 0
  BROKEN_SYMLINK: 0
  EXTERNAL: 0
  INSTRUMENTED: 3
  MISSING: 0
  NOT_A_TEST: 0
  SIMPLE: 0
  

---