We’ve all heard the old saying “A picture is worth a thousand words” (in German, “Ein Bild sagt mehr als tausend Worte”). It’s trite, but it’s true. An image can cut through the crap and get everyone on the same page in an instant. In the software development world, examples carry the same power. That’s why I, like so many others, am a big fan of Specification by Example.

Using terms from object-oriented programming, I’m aware that examples don’t fully describe rules – they are an “instance” of a rule. However, like images, examples make business logic tangible. That’s why they’re so great at helping team members to reach a mutual understanding. And – wait a minute – isn’t that exactly what functional test cases are? “Instances” of rules?

Specification by Example, 10 Years Later

I’ve been watching the Specification by Example community pretty closely these days, and I was excited to stumble upon Gojko Adzic’s great article “Specification by Example, 10 years later.” Gojko, who literally wrote the book on Specification by Example ten years ago, ran an extensive research project predominantly focused on teams who use examples to capture acceptance criteria. There are tons of interesting findings there—I encourage you to give the complete write-up a close read.

I’d like to speak to a few points that really stood out to me…

Confirmation That Using Examples Boosts Product Quality

If you talk to teams using Specification by Example approaches, you’ll almost certainly hear anecdotal evidence that it has helped them improved product quality. Gojko’s study quantifies this with compelling statistics. He asked 500+ participants to estimate the quality of their releases based on the frequency and severity of production problems. He then segmented these results based on whether or not teams used examples as acceptance criteria. Those that were using acceptance criteria were nearly 3X more likely to report “great” quality than teams that did not.

The Given-When-Then Format Reigns Supreme

The study also found that “Given-When-Then” dominates example formats, by far. It’s used by 71% of the respondents. I believe this is because it’s the best balance of expressiveness and developer productivity. The study also noted that tables worked well for capturing large numbers of examples with similar structure. However, tables and Given-When-Then don’t have to be mutually exclusive. It’s important to note that tables have found their way into BDD Scenario Outlines, too.

#agile adoption #tdd #agile testing #bdd #test