How Google TestsSoftware - The Life of a SET

SETs are Software Engineersin Test. They are software engineers who happen to write testing functionality.First and foremost, SETs are developers and the role is touted as a 100% codingrole in our recruiting literature and internal job promotion ladders. When SETcandidates are interviewed, the “coding bar” is nearly identical to the SWErole with more emphasis that SETs know how to test the code they create. Inother words, both SWEs and SETs answer coding questions. SETs are expected tonail a set of testing questions as well.

As you might imagine, it isa difficult role to fill and it is entirely possible that the low numbers of SETsisn’t because Google has created a magic formula for productivity but more of aresult of adapting our engineering practice around the reality that the SETskill set is really hard to find. We optimize on this very important task andbuild processes around the people who do it.

It is usually the case thatSETs are not involved early in the design phase. Their exclusion is not so muchpurposeful as it is a by-product of how a lot of Google projects are born. Acommon scenario for new project creation is that some informal 20% effort takesa life of its own as an actual Google branded product. Gmail and Chrome OS areboth projects that started out as ideas that were not formally mandated byGoogle but over time grew into shipping products with teams of developers andtesters working on them. In such cases early development is not about quality,it is about proving out a concept and working on things like scale andperformance that must be right before quality could even be an issue. If youcan't build a web service that scales, testing is not your biggest problem!

Once it is clear that aproduct can and will be built and shipped, that's when the development teamseeks out test involvement.

You can imagine a processlike this: someone has an idea, they think about it, write experimental code,seek out opinions of others, write some more code, get others involved, writeeven more code, realize they are onto something important, write more code tomold the idea into something that they can present to others to get feedback... somewhere in all this an actual project is created in Google's projectdatabase and the project becomes real. Testers don't get involved until itbecomes real.

Do all real projects gettesters? Not by default. Smaller projects and those meant for limited usersoften get tested exclusively by the people who build it. Others that areriskier to our users or the enterprise (much more about risk later) get testingattention.

The onus is on thedevelopment teams to solicit help from testers and convince them that theirproject is exciting and full of potential. Dev Directors explain their project,progress and ship schedule to Test Directors who then discuss how the testingburden is to be shared and agree on things like SWE involvement in testing,expected unit testing levels and how the duties of the release process aregoing to be shared. SETs may not be involved at project inception, but once theproject becomes real we have vast influence over how it is to be executed.

And when I say"testing" I don't just mean exercising code paths. Testers might notbe involved from the beginning ... but testing is. In fact, an SET's impact isfelt even before a developer manages to check code into the build. Stay tunedto understand what I am talking about.