性能测试工具 nGrinder 项目剖析及二次开发

转：https://testerhome.com/topics/4225

0.背景

组内需要一款轻量级的性能测试工具，之前考虑过LR（太笨重，单实例，当然它的地位是不容置疑的），阿里云的PTS(https://pts.aliyun.com/lite/index.htm，仅支持阿里云内网和公网机器)，Gatling(http://gatling.io/#/)没有TPS数据等等，不太适合我们。

nGrinderr是NAVER（韩国最大互联网公司NHN旗下搜索引擎网站）开源的性能测试工具，直接部署成web服务，支持多用户使用，可扩展性好，可自定义plugin(http://www.cubrid.org/wiki_ngrinder/entry/how-to-develop-plugin)，wiki文档较丰富(http://www.cubrid.org/wiki_ngrinder/entry/ngrinder-devzone)，数据及图形化展示满足需求；但是展示的统计数据较简单，二次开发整合数据：TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数字段，并将这些数据展示在详细测试报告页中。

1.项目剖析

1-1. nGrinder架构

nGrinder是一款在一系列机器上执行Groovy或JPython测试脚本的应用，内部引擎是基于Grinder。
架构图：

层级图：

默认的NGRINDER_HOME为/root/.ngrinder，大多是配置文件和数据文件。

目录/root/.ngrinder/perftest/0_999下，以每个test_id为名的文件夹对应的存储了执行性能测试时的采样数据：

*.data文件就是执行性能测试时对应的各种性能采样数据，性能测试详细报告页就是根据这些data文件，进行图形化展示（ajax）。

nGrinder包含2大组件：
1）Controller
为性能测试提供web interface
协同测试进程
收集和显示测试数据
新建和修改脚本

2）Agent
agent mode: 运行进程和线程，压测目标服务
monitor mode: 监控目标系统性能(cpu/memory)，可以自定义收集的数据(比如 jvm数据)

http://www.cubrid.org/wiki_ngrinder/entry/general-architecture

1-2. 技术栈

1)Controller 层
FreeMarker: 基于Java的模板引擎
Spring Security
Spring Mvc：Spring MVC provides rich functionality for building robust web applications.
GSon
SVNKit Dav

2)Service 层
Grinder
Spring
EhCache: Ehcache has excellent Spring integration.

3)Data层
Spring Data
Hibernate:Hibernate is a powerful technology for persisting data,and it is Spring Data back-end within nGrinder.
H2: (nGrinder默认使用该DB)
Cubrid:(nGrinder同一家公司的DB)
Liquidase: Liquibase is an open source that automates database schema updates.
SVNKit

http://www.cubrid.org/wiki_ngrinder/entry/technology-stack

2.源码实现

需求：在详细测试报告页中展示TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数这些数据。

修改Controller层，增加数据处理业务逻辑(计算TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数)

在获取采样数据
ngrinder-core/src/main/java/net/grinder/SingleConsole.java中新增处理业务逻辑，核心修改代码片段：

    // tps list

    List<Double> tps = new CopyOnWriteArrayList<Double>();

    // rt list

    List<Double> meanTestTime = new CopyOnWriteArrayList<Double>();

    /**

     *

     * 每次请求调用一次 Build up statistics for current sampling.

     *

     * @param accumulatedStatistics

     *            intervalStatistics

     * @param intervalStatistics

     *            accumulatedStatistics

     */

    protected void updateStatistics(StatisticsSet intervalStatistics,

            StatisticsSet accumulatedStatistics) {

        Map<String, Object> result = newHashMap();

        result.put("testTime", getCurrentRunningTime() / 1000);

        List<Map<String, Object>> cumulativeStatistics = new ArrayList<Map<String, Object>>();

        List<Map<String, Object>> lastSampleStatistics = new ArrayList<Map<String, Object>>();

        for (Test test : accumulatedStatisticMapPerTest.keySet()) {

            Map<String, Object> accumulatedStatisticMap = newHashMap();

            Map<String, Object> intervalStatisticsMap = newHashMap();

            StatisticsSet accumulatedSet = this.accumulatedStatisticMapPerTest

                    .get(test);

            StatisticsSet intervalSet = this.intervalStatisticMapPerTest

                    .get(test);

            accumulatedStatisticMap.put("testNumber", test.getNumber());

            accumulatedStatisticMap.put("testDescription",

                    test.getDescription());

            intervalStatisticsMap.put("testNumber", test.getNumber());

            intervalStatisticsMap.put("testDescription", test.getDescription());

            // When only 1 test is running, it's better to use the parametrized

            // snapshot.

            for (Entry<String, StatisticExpression> each : getExpressionEntrySet()) {

                if (INTERESTING_STATISTICS.contains(each.getKey())) {

                    accumulatedStatisticMap.put(

                            each.getKey(),

                            getRealDoubleValue(each.getValue().getDoubleValue(

                                    accumulatedSet)));

                    intervalStatisticsMap.put(

                            each.getKey(),

                            getRealDoubleValue(each.getValue().getDoubleValue(

                                    intervalSet)));

                }

            }

            cumulativeStatistics.add(accumulatedStatisticMap);

            lastSampleStatistics.add(intervalStatisticsMap);

        }

        Map<String, Object> totalStatistics = newHashMap();

        for (Entry<String, StatisticExpression> each : getExpressionEntrySet()) {

            if (INTERESTING_STATISTICS.contains(each.getKey())) {

                totalStatistics.put(each.getKey(), getRealDoubleValue(each

                        .getValue().getDoubleValue(accumulatedStatistics)));

            }

        }

        LOGGER.debug("hugang start get plug data");

        // 获取tps, rt集合

        for (Entry<String, StatisticExpression> each : getExpressionEntrySet()) {

            if ("TPS".equals(each.getKey())) {

                tps.add((Double) getRealDoubleValue(each.getValue()

                        .getDoubleValue(intervalStatistics)));

            } else if ("Mean_Test_Time_(ms)".equals(each.getKey())) {

                meanTestTime.add((Double) getRealDoubleValue(each.getValue()

                        .getDoubleValue(intervalStatistics)));

            }

        }

        result.put("totalStatistics", totalStatistics);

        result.put("cumulativeStatistics", cumulativeStatistics);

        result.put("lastSampleStatistics", lastSampleStatistics);

        result.put("tpsChartData", getTpsValues());

        result.put("peakTpsForGraph", this.peakTpsForGraph);

        synchronized (this) {

            result.put(GrinderConstants.P_PROCESS, this.runningProcess);

            result.put(GrinderConstants.P_THREAD, this.runningThread);

            result.put("success", !isAllTestFinished());

        }

        // Finally overwrite.. current one.

        this.statisticData = result;

    }

    /**

     * 从updateStatistics()累加数据, list :rt 和 tps, 为成员变量

     *

     * 再处理集合，放到statisticData中

     *

     * @author hugang

     */

    public void getPlusResult(){

        LOGGER.debug("hugang getPlusResult() tpslist {}  rtlist is {}",

                tps.toString(), meanTestTime.toString());

        int i = 0;

        int j = 0;

        // list转成数组， 标准库使用数组作为参数

        double[] tpsArray = new double[tps.size()];

        for (double tpsNum : tps) {

            tpsArray[i++] = tpsNum;

        }

        // list转成数组

        double[] meanTestTimeArray = new double[meanTestTime.size()];

        for (double meanTime : meanTestTime) {

            meanTestTimeArray[j++] = meanTime;

        }

        // tps 标准差

        double tpsStd = new StandardDeviation().evaluate(tpsArray);

        // tps 平均值

        double tpsMean = new Mean().evaluate(tpsArray, 0, tpsArray.length);

        // tps 波动率= tps 标准差 / tps 平均值

        double tpsVix = 0;

        if(0 != tpsMean){

            tpsVix = tpsStd / tpsMean;

        }

        // meanTestTime 百分位数

        Percentile percentile = new Percentile();

        // 先排序

        Arrays.sort(meanTestTimeArray);

        // meanTestTime最小值

        double minMeanTime = meanTestTimeArray[0];

        double twentyFiveMeanTime = percentile.evaluate(meanTestTimeArray, 25);

        double fiftyMeanTime = percentile.evaluate(meanTestTimeArray, 50);

        double serventyFiveMeanTime = percentile

                .evaluate(meanTestTimeArray, 75);

        double eightyMeanTime = percentile.evaluate(meanTestTimeArray, 80);

        double eightyFiveMeanTime = percentile.evaluate(meanTestTimeArray, 85);

        double ninetyMeanTime = percentile.evaluate(meanTestTimeArray, 90);

        double ninetyFiveMeanTime = percentile.evaluate(meanTestTimeArray, 95);

        double ninetyNineMeanTime = percentile.evaluate(meanTestTimeArray, 99);

        int length = meanTestTimeArray.length;

        // meanTestTime最高值

        double maxMeanTime = meanTestTimeArray[length - 1];

        // meanTestTime平均值

        // double TimeMean = new Mean().evaluate(meanTestTimeArray, 0,

        // meanTestTimeArray.length);

        LOGGER.debug(

                "hugang plug Statistics MinMeanTime {}  MaxMeanTime is {}",

                minMeanTime, maxMeanTime);

        // 附加信息 hugang

        // tps 标准差, tps 波动率， 最小/最大RT, RT百分位数

        Map<String, Object> plusStatistics = newHashMap();

        plusStatistics.put("tpsStd", tpsStd);

//      plusStatistics.put("tpsMean", tpsMean);

        plusStatistics.put("tpsVix", tpsVix);

        plusStatistics.put("minMeanTime", minMeanTime);

        plusStatistics.put("twentyFiveMeanTime", twentyFiveMeanTime);

        plusStatistics.put("fiftyMeanTime", fiftyMeanTime);

        plusStatistics.put("serventyFiveMeanTime", serventyFiveMeanTime);

        plusStatistics.put("eightyMeanTime", eightyMeanTime);

        plusStatistics.put("eightyFiveMeanTime", eightyFiveMeanTime);

        plusStatistics.put("ninetyMeanTime", ninetyMeanTime);

        plusStatistics.put("ninetyFiveMeanTime", ninetyFiveMeanTime);

        plusStatistics.put("ninetyNineMeanTime", ninetyNineMeanTime);

        plusStatistics.put("maxMeanTime", maxMeanTime);

        LOGGER.debug("SingleConsole plug Statistics map plusStatistics {}", plusStatistics);

        this.statisticData.put("plusStatistics", plusStatistics);

    }

    /**

     *

     * 停止采样数据

     * Stop sampling.

     */

    public void unregisterSampling() {

        this.currentNotFinishedProcessCount = 0;

        if (sampleModel != null) {

            this.sampleModel.reset();

            this.sampleModel.stop();

        }

        LOGGER.info("Sampling is stopped");

        informTestSamplingEnd();

        // 结束采样后，处理数据

        // hugang

        getPlusResult();

    }

Map statisticData为不同数据集集合。

Service层从SingleConsole类中获取数据集statisticData：
ngrinder-controller/src/main/java/org/ngrinder/perftest/server/PerfTestService.java 中Map<String, Object> result = consoleManager.getConsoleUsingPort(perfTest.getPort()).getStatisticsData();



/**

     * Update the given {@link PerfTest} properties after test finished.

     *

     * @param perfTest perfTest

     *

     * getConsoleUsingPort()获取数据

     *

     *

     * hugang

     */

    public void updatePerfTestAfterTestFinish(PerfTest perfTest) {

        checkNotNull(perfTest);

        Map<String, Object> result = consoleManager.getConsoleUsingPort(perfTest.getPort()).getStatisticsData();

        @SuppressWarnings("unchecked")

        Map<String, Object> totalStatistics = MapUtils.getMap(result, "totalStatistics", MapUtils.EMPTY_MAP);

        // 获取附加数据

        Map<String, Object> plusStatistics = MapUtils.getMap(result, "plusStatistics", MapUtils.EMPTY_MAP);

        LOGGER.info("Total Statistics for test {}  is {}", perfTest.getId(), totalStatistics);

        LOGGER.info("plug Statistics for test {}  is {}", perfTest.getId(), plusStatistics);

        perfTest.setTps(parseDoubleWithSafety(totalStatistics, "TPS", 0D));

        perfTest.setMeanTestTime(parseDoubleWithSafety(totalStatistics, "Mean_Test_Time_(ms)", 0D));

        perfTest.setPeakTps(parseDoubleWithSafety(totalStatistics, "Peak_TPS", 0D));

        perfTest.setTests(MapUtils.getDouble(totalStatistics, "Tests", 0D).longValue());

        perfTest.setErrors(MapUtils.getDouble(totalStatistics, "Errors", 0D).longValue());

        // 附加信息写到model， 持久化

        perfTest.setTpsStd(parseDoubleWithSafety(plusStatistics, "tpsStd", 0D));

        perfTest.setTpsVix(parseDoubleWithSafety(plusStatistics, "tpsVix", 0D));

        perfTest.setMinRT(parseDoubleWithSafety(plusStatistics, "minMeanTime", 0D));

        perfTest.setTwentyFiveMeanTime(parseDoubleWithSafety(plusStatistics, "twentyFiveMeanTime", 0D));

        perfTest.setFiftyMeanTime(parseDoubleWithSafety(plusStatistics, "fiftyMeanTime", 0D));

        perfTest.setServentyFiveMeanTime(parseDoubleWithSafety(plusStatistics, "serventyFiveMeanTime", 0D));

        perfTest.setEightyMeanTime(parseDoubleWithSafety(plusStatistics, "eightyMeanTime", 0D));

        perfTest.setEightyFiveMeanTime(parseDoubleWithSafety(plusStatistics, "eightyFiveMeanTime", 0D));

        perfTest.setNinetyMeanTime(parseDoubleWithSafety(plusStatistics, "ninetyMeanTime", 0D));

        perfTest.setNinetyFiveMeanTime(parseDoubleWithSafety(plusStatistics, "ninetyFiveMeanTime", 0D));

        perfTest.setNinetyNineMeanTime(parseDoubleWithSafety(plusStatistics, "ninetyNineMeanTime", 0D));

        perfTest.setMaxRT(parseDoubleWithSafety(plusStatistics, "maxMeanTime", 0D));

    }

修改Model层，在javabean中增加TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数, JPA持久化（H2 DB新增TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数字段)

model文件为：ngrinder-core/src/main/java/org/ngrinder/model/PerfTest.java



    /**

     * 新增字段，TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数

     * hugang

     */

    @Expose

    @Column(name = "tpsStd")

    private Double tpsStd;

    @Expose

    @Column(name = "tpsVix")

    private Double tpsVix;

    @Expose

    @Column(name = "minRT")

    private Double minRT;

    @Expose

    @Column(name = "twentyFiveMeanTime")

    private Double twentyFiveMeanTime;

    @Expose

    @Column(name = "fiftyMeanTime")

    private Double fiftyMeanTime;

    @Expose

    @Column(name = "serventyFiveMeanTime")

    private Double serventyFiveMeanTime;

    @Expose

    @Column(name = "eightyMeanTime")

    private Double eightyMeanTime;

    @Expose

    @Column(name = "eightyFiveMeanTime")

    private Double eightyFiveMeanTime;

    @Expose

    @Column(name = "ninetyMeanTime")

    private Double ninetyMeanTime;

    @Expose

    @Column(name = "ninetyFiveMeanTime")

    private Double ninetyFiveMeanTime;

    @Expose

    @Column(name = "ninetyNineMeanTime")

    private Double ninetyNineMeanTime;

    @Expose

    @Column(name = "maxRT")

    private Double maxRT;

    对应的set(), get()

还需修改db change文件(因为系统DB默认使用H2，只需修改H2对应的xml)，ngrinder-controller/src/main/resources/ngrinder_datachange_logfile/db.changelog_schema_H2.xml



create table PERF_TEST (

            id bigint generated by default as identity unique,

            created_date timestamp,

            last_modified_date timestamp,

            agent_count integer,

            description varchar(2048),

            distribution_path varchar(255),

            duration bigint,

            errors integer,

            finish_time timestamp,

            ignore_sample_count integer,

            init_processes integer,

            init_sleep_time integer,

            last_progress_message varchar(2048),

            mean_test_time double,

            peak_tps double,

            errorRate double,

            tpsStd double,

            tpsVix double,

            minRT double,

            twentyFiveMeanTime double,

            fiftyMeanTime double,

            serventyFiveMeanTime double,

            eightyMeanTime double,

            eightyFiveMeanTime double,

            ninetyMeanTime double,

            ninetyFiveMeanTime double,

            ninetyNineMeanTime double,

            maxRT double,

系统重启加载时，Liquidase会自动更新DB。

修改View层，在详细报告对应的freemarker模板新增TPS标准差，TPS波动率，最小/大RT，RT 25/50/75/80/85/90/95/99百分位数字段，前端新增展示这些数据

ngrinder-controller/src/main/webapp/WEB-INF/ftl/perftest/detail_report.ftl



                <#-- hugang -->

                <#-- 新增 错误率，TPS标准差，TPS波动率，最小RT, 最大RT, RT 25/50/75/80/85/90/95/99百分位数 -->

                <tr>

                    <th><@spring.message "perfTest.report.errorRate"/></th>

                    <td>${(test.errors /(test.tests + test.errors))!""}</td>

                </tr>

                <tr>

                    <th><@spring.message "perfTest.report.tpsStd"/></th>

                    <td>${test.tpsStd!""}</td>

                </tr>

                <tr>

                    <th><@spring.message "perfTest.report.tpsVix"/></th>

                    <td>${test.tpsVix!""}</td>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.minRT"/></th>

                    <td>${test.minRT!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.TwentyFiveMeanTime"/></th>

                    <td>${test.twentyFiveMeanTime!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.FiftyMeanTime"/></th>

                    <td>${test.fiftyMeanTime!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.ServentyFiveMeanTime"/></th>

                    <td>${test.serventyFiveMeanTime!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.EightyMeanTime"/></th>

                    <td>${test.eightyMeanTime!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.EightyFiveMeanTime"/></th>

                    <td>${test.eightyFiveMeanTime!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.NinetyMeanTime"/></th>

                    <td>${test.ninetyMeanTime!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.NinetyFiveMeanTime"/></th>

                    <td>${test.ninetyFiveMeanTime!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.NinetyNineMeanTime"/></th>

                    <td>${test.ninetyNineMeanTime!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

                    </tr>

                    <tr>

                    <th><@spring.message "perfTest.report.maxRT"/></th>

                    <td>${test.maxRT!""}&nbsp;&nbsp; <code>ms</code></td>

                </tr>

还有个坑，就是从github拉下的代码，源码中pom.xml依赖的jar包不完整，直接打不了包，项目有的依赖的jar 公有maven仓库已经没有了，需要自己从网上找jar包，安装到本地仓库，我归整了下：

http://download.csdn.net/detail/neven7/9443895

直接在ngrinder根路径下执行打包命令：

mvn -Dmaven.test.skip=true clean package

部署生成的war即可。

3.结果展示

在详细报告页新增如下数据结果：