Automating CSS Regression Testing
The following is a guest post by Garris Shipon . We've touched on the four types of CSS testing here before. Regression testing is the hardest. It's the type where you're trying to test if a change you made to CSS resulted in any unexpected visual problems. This is made more difficult with responsive designs. Garris built a tool for doing this as he embarked upon a new responsive design for a large scale site. Here, he'll walk you through the whole thing.
Why CSS testing?
Do a search for CSS regression testing and a common theme becomes clear – breaking CSS is easy, testing it is hard. This was the case at the onset of a large responsive refactoring project I scoped earlier this year for Art.com.
Like many other established web companies, Art.com is in the process of adding responsive behavior to its massive e-commerce web app, which was originally designed for 1024px desktop screens.
I realized this would be a big and delicate job. Adding multiple breakpoint behaviors which (by their very nature) are mostly off-screen at any point in time means we would have a lot of hard-to-find errors. We needed a way for our engineers to automate bug discovery before slamming our QA process with hundreds of ticky-tacky little layout issues (which would all need to be documented, tracked and fixed).
CSS testing options
Looking at existing options for CSS regression coverage, I discovered that most came with a lot of integration overhead. To me, these solutions seem more geared for QA engineers who are focused exclusively on testing. These testing apps have deep and wide coverage potential but are too complex and bulky for most UI engineers to easily integrate into their development process.
The solution I wanted would be easy to install locally, use familiar syntax and give us a reasonable amount of confidence that a selector change I make for mobile isn't going to result in a hard-to-find bug in my desktop layout.
Of the existing solutions, the one which had the most potential for my needs was phantomCSSwritten by James Cryer . This is an awesome tool for visually testing complex UI module interaction states, but it requires full-on CasperJS scripting which is still unnecessarily cumbersome for testing large sets of static CMS templates at different screen dimensions.
This situation was the impetus for creating BackstopJS , which is essentially a wrapper forResembleJS (also written by James Cryer ) that enables an easy-to-configure visual regression test matrix across multiple URLs, page elements and screen sizes.
The following is a 15 minute walk-through of an installation and initial configuration of BackstopJS.
Installation
BackstopJS can be downloaded as a .zip file from GitHub, but it's best to install it using the super useful Bower package manager . The rest of the instructional will be based on the Bower install and a simple demo project (download zip), which is taken directly directly the Bootstrap example page .
Node and the NPM package manager are foundational dependencies also required for the rest of the install. It's possible you already have these installed (from other projects). If not, I'd really recommend you to learn more . There are really amazing developments going on in this ecosystem!
Expand the simple demo
Lets start by downloading the demo project files . Or, you can use your own active project. We will install the testing framework right into here:
Install BackstopJS with Bower & NPM
Go to your project root, in this case: myCoolProject
$ cd ~/path-to-myProjects/myCoolProject
$ bower install backstopjs
Your directory should look like this:
Install dependencies
So far so good, you now have BackstopJS scaffolding, but there are some super useful packages that BackstopJS depends on so we need to install those.
Install local dependencies with NPM
Node and the NPM package manager and required for the rest of the install.
$ cd bower_components/backstopjs
$ npm install
If you don't already have a global Gulp instance...
Gulp is a great system for enabling various task automation. It's great for project building, asset minification, CSS preprocessing etc. It is a more functional way of tackling the kinds of tasks Grunt became popular for.
$ sudo npm install gulp -g
If you don't already have a global PhantomJS install...
Well then, this is your lucky day! PhantomJS is a headless web browser. It does everything a web browser does, except display content to the screen.
$ sudo npm install phantomjs
If you don't already have a global CasperJS install...
Then don't miss this. CasperJS ( installation docs ) does for headless web clients what jQuery did for DOM selection.
$ sudo npm install -g casperjs
Install complete!
Still with me? If you have made it this far then you have (among other things) some badass CSS regression testing ingredients. Lets see what we can do.
Generate a test configuration template
From the myCoolProject/bower_components/backstopjs
directory:
$ gulp genConfig
This will generate a boilerplate JSON configuration file for you at the root of your project e.g. `myCoolProject/backstop.json`. This is where you can specify the URL's, selectors to test and at which breakpoints you want to test them at.
{
"viewports" : [
{
"name": "phone",
"viewport": {"width": 320, "height": 480}
}
,{
"name": "tablet_v",
"viewport": {"width": 568, "height": 1024}
}
,{
"name": "tablet_h",
"viewport": {"width": 1024, "height": 768}
}
]
,"grabConfigs" : [
{
"testName":"http://getbootstrap.com"
,"url":"http://getbootstrap.com"
,"hideSelectors": [
]
,"removeSelectors": [
"#carbonads-container"
]
,"selectors":[
"header"
,"main"
,"body .bs-docs-featurette:nth-of-type(1)"
,"body .bs-docs-featurette:nth-of-type(2)"
,"footer"
,"body"
]
}
]
}
In this configuration you can see three viewports
objects, one for phone , tablet vertical andtablet horizontal , each with a name
and a viewport
property. You can add as manyviewports
objects as you need. BackstopJS requires at least one .
Then we have grabConfigs
. This specifies the URLs and element selectors that BackstopJS will test. The selectors generally accept standard CSS selector notation. It's useful to think of everygrabConfigs
object as a test for a specific static page or global app state. Here too, you can add as many grabConfigs
objects as you need. BackstopJS requires at least one .
You may notice that in this config, our URL is pointing to http://getbootstrap.com (the homepage of the Bootstrap design component library) and that is what we would see if we were to run our test now. This is here to illustrate that BackstopJS can point to local or remote URLs so it's easy to imagine repurposing the same tests for local development, QA, staging and production environments.
Modify the configuration template
For our demo we want to test our local project file, so please make the following change replacing the grabConfigs
node in `myCoolProject/backstop.json`.
,"grabConfigs" : [
{
"testName":"http://getbootstrap.com"
,"url":"../../index.html"
,"hideSelectors": [
]
,"removeSelectors": [
]
,"selectors":[
"nav"
,".jumbotron"
,"body .col-md-4:nth-of-type(1)"
,"body .col-md-4:nth-of-type(2)"
,"body .col-md-4:nth-of-type(3)"
,"footer"
]
}
]
generating (or updating) reference screenshots
From the `myCoolProject/bower_components/backstopjs` directory:
$ gulp reference
This task will create a (or update an existing) bitmaps_reference
directory with screen captures representing all specified selectors at every breakpoint. Take a look inside `bower_components/backstopjs/bitmaps_reference`:
So far so good. We now have our reference set. Now lets run a test!
Running our first test
We are about to run our first test. But keep in mind, we haven't changed anything in our project yet. So our tests should pass.
From the `myCoolProject/bower_components/backstopjs` directory:
$ gulp test
This task will create a new set of images in `myCoolProject/bower_components/backstopjs/bitmaps_test/<timestamp>/`.
Once the test images are generated, a web browser should open and a report comparing the most recent test bitmaps against the current reference images will run. Significant differences will be detected and shown.
Test screen shots should have been generated and a clean test report should be displayed. Now let's try changing our CSS and see what happens.
Updating our index file and running our second test
Open our test file at `myCoolProject/index.html` and add the following code somewhere in the<head>
tag of the markup:
<style>
.jumbotron {
padding: 0px;
}
</style>
Now, From the `myCoolProject/bower_components/backstopjs` directory.
$ gulp test
Our test should run again and errors should be found, scroll the report down to see a visual diff of the problems.
Our visual diff contains the reference capture, the most recent test capture and the visual diff file.
And there you have it – regression found!
This is a very simple example. In real life, designers and engineers may find themselves working on very complex responsive layouts. That is when a system like this really provides value. By automating the repetitive tasks we can move our CSS projects forward by focusing on more complex and creative tasks that can't be automated.
There are a lot of things I can imagine for BackstopJS to satisfy more use cases, here are a few obvious ones I would love to have for myself:
- A UI for managing multiple config profiles and associated reference data.
- Firefox and IE rendering
- Currently, screenshots are executed when the native document ready event is fired, it would be more useful if we could specify a custom ready event for things like web apps that may be making Ajax calls or running some other async process before the display can be fully rendered.
- A much easier install process
Automating CSS Regression Testing的更多相关文章
- Web自动化测试工具调研
背景 Web自动化测试越来越被重视, 因为现在Web已经是工程化的状态. 如何通过工具测试, 保证Web开发的质量,提升开发效率,是Web工具的诞生的来由. Web测试分为以下几个方面: 1. 界面测 ...
- Frontend Development
原文链接: https://github.com/dypsilon/frontend-dev-bookmarks Frontend Development Looking for something ...
- 2015年最佳的12个 CSS 开发工具推荐
CSS所能做的就是改变网页的布局.排版和调整字间距等,但编写 CSS 并不是一项容易的任务,当你接触新的 CSS3 属性及其各自的浏览器前缀的时候,你会发现很伤脑经.值得庆幸的是一些优秀的开发人员提供 ...
- 用CSS制作伪标签云
performance testing stress testing conformance testing acceptane testing smoke testing regression te ...
- Testing - Tips
1 --- 冒烟测试.可用性测试和回归测试的区别? 在测试领域中,冒烟测试(smoke test).可用性测试(sanity test)和回归测试(regression test)彼此之间很相似,范围 ...
- CSS样式汇总
1. Overflow: 是否隐藏超出容器范围之外的内容,主要参数包括Hidden(隐藏),Auto(根据容器内容自动显示滚动条),scroll(显示滚动条,即使内容不超出容器范围,也会显示一个边框, ...
- Testing - 软件测试知识梳理 - 相关词汇
测试策略 描述测试工程的总体方法和目标:根据测试需求,描述在什么测试阶,依据什么测试要素和目标,进行什么种类的测试,使用什么样的测试方法和工具. 测试策略的制定主要包含如下内容: 确定测试过程要使用的 ...
- CSS选择器的优化
前面花了几个篇幅着重介绍了CSS的选择器的使用,我将其分成三个部分:CSS基本选择器.CSS属性选择器以及CSS伪类选择器.那么今天我主要想和大家一起来学习——CSS选择器方面的性能优化.因为对性能这 ...
- FW:Software Testing
Software Testing Testing with a Purpose Software testing is performed to verify that the completed s ...
随机推荐
- socketserver源码简介
一.socketserver流程简介 +------------+ | BaseServer | +------------+ | v +-----------+ +----------------- ...
- js排序算法01——冒泡排序
在codewars上面刷题卡住刷不下去了,意识到自己算法方面的不足,准备写一些算法方面的文章,此为一. 冒泡排序是很常见简单的算法了,每次比较任何两个相邻的项,如果第一个比第二个大,则交换他们,就像气 ...
- vue本地项目静态资源管理
vue本地项目静态资源管理 统一放在src/static里面css,images,js index.html里面使用重置样式 <link rel="stylesheet" h ...
- C# 处理 JSON 常用的帮助类
C#请求接口的方法,具体代码: 首先需要添加引用和第三方的组件,具体如下: 引用命名空间: using System.IO; using Newtonsoft.Json.Linq; using Sys ...
- Poj 3318 Matrix Multiplication( 矩阵压缩)
Matrix Multiplication Time Limit: 2000MS Memory Limit: 65536K Total Submissions: 18928 Accepted: ...
- ElementTree之Xml文档处理
ElementTree: 表示整个XML层级结构 Element: 表示树形结构中所有的父节点 SubElement: 表示树形结构中所有的子节点 有些节点既是父节点,又是子节点 下面来看下这两个类的 ...
- UML_03_类图
一.前言 类图是UML结构图,在类和接口的层次上显示设计系统的结构,显示它们的特性.约束和关系等,是定义其它图的基础. 二.类图 如上图,在类图中表示方法如下: 斜体 :抽象类.抽象方法 下划线 :静 ...
- 如何修改MyEclipse的默认编码方式
在创建jsp页面时,默认首行出现“<%@ page language="java" import="java.util.*" pageEncoding=& ...
- 桶排序bucket sort
桶排序 (Bucket sort)或所谓的箱排序的原理是将数组分到有限数量的桶子里,然后对每个桶子再分别排序(有可能再使用别的排序算法或是以递归方式继续使用桶排序进行排序),最后将各个桶中的数据有序的 ...
- C#中正则表达式编程(未完,待补充)
对于只存储一个匹配,可用Match类: 一般模式: Regex reg = new Regex(string pattern); string str = "###############& ...