BDD is bigger than I thought

… so now what am I thinking?

Introduction — tests that makes me happy

One of my favourite things about joining digital product consultancy White October six years ago was discovering Behaviour-Driven Development (BDD). As a developer, the satisfaction of seeing descriptions of users’ actual behaviour with an application scroll past in green was hard to beat…

...Then I should see a thanks messageAnd my account should be credited with 500 pointsAnd Sam should feel like an awesome developer because his code Officially Works

You get the picture.

Gradually, I decided I wanted to learn more about this BDD thing. Much of my initial learning was around tooling, but recently I’ve done two things that have really opened my eyes:

• I read BDD in Action by John Ferguson Smart (with an introduction by Dan North, the creator of BDD)
• I watched 10 years of Doing Behaviour-Driven Development All Wrong by Liz Keogh, co-creator of JBehave

What I discovered made me realise that I didn’t actually know very much about BDD at all…

BDD starts earlier than I thought

Here’s a question: What’s the “main thing” about BDD?

If you said “automated tests that simulate real user behaviour”, you’re just like I was until recently.

But here’s “BDD in a tweet” from its creator Dan North:

And here’s an oft-quoted line from Liz Keogh:

…having conversations is more important than capturing conversations is more important than automating conversations.

In the conference talk I mentioned by Liz Keogh, she even reports a conversation with Dan along the lines of “it’s a shame we made it so easy to create automated tests from examples, because now everyone thinks that’s what BDD is about”.

Guilty as charged! I was aware of some of the broader implications of BDD, but wasn’t seriously engaged with them. Writing executable specifications and creating automated tests from a user perspective is incredibly valuable, and I’m definitely going to keep doing those things, but there’s more to BDD than that.

So, if BDD is actually a process for the full software-development lifecycle, what else does it cover? Here are some of the bigger pieces of the puzzle:

Focus on business goals

BDD brings a set of project principles to the table, and includes techniques to ensure you’re following those principles. These techniques and principles come into play from the beginning of a software project, way before you start talking about examples or scenarios — the BDD process starts earlier than I thought!

One of the key BDD principles is to focus on business goals, and to only work on features that will help achieve those business goals. BDD pioneer Chris Matts invented a process called Feature Injection to help you do this:

Essentially, you start by working out your business goals, and then from those you “inject the features”, proactively looking for features that will deliver those business goals. This is fundamentally different from devising a big backlog of stories and then trying to prioritise them in relation to goals. It’s also different from User Story Mapping, which arranges your stories to walk you through a first version of the system. With feature injection, everything flows from the goals, rather than the goals being simply a guiding principle when working with stories you derived when thinking about a system.

The third stage of Feature Injection is to “spot the examples”, using high-level examples to flesh out the missing scope of the features identified.

(Whilst this suggests that there are only 3 “levels” of description of a system — goals, features and examples — BDD actually defines a fuller set of levels which go all the way down to code and include other steps in between. Read BDD in Action for more on this.)

Another process which BDD practitioners use to keep this explicit link to business goals is Impact Mapping, invented by Gojko Adzic. You can read more about it at https://www.impactmapping.org/index.html

Embrace uncertainty and change

Two more important BDD principles are Real Options and Deliberate Discovery. These allow you to embrace uncertainty and change in your projects.

In short, Real Options is the idea that you shouldn’t commit to a solution until the “last responsible moment”, waiting until you have maximum possible certainty that it’s the right one. But this doesn’t mean putting of decisions forever — decisions should be at the last responsible moment, not just the last moment!

Deliberate Discovery is the other side of this coin, taking seriously your own ignorance about which solution to a problem is the best. In a nutshell, Deliberate Discovery says that you should aim to identify and reduce uncertainty by proactively investigating areas of ignorance about a possible solution.

You can read more about Feature Injection, Real Options and Deliberate Discovery here: https://lizkeogh.com/2012/06/01/bdd-in-the-large/

Examples expressed in scenarios are an aid to conversation, which is an aid to understanding.

Finally, a broader point, which also links back to the scenarios that I started my BDD journey with:

The key reason for creating scenarios (structured natural-language descriptions of user actions) is not so that you can created automated tests from them. In BDD, the key reason is the conversations that happen when a developer creates those scenarios together with a Business Analyst and a Tester (a “Three Amigos” session). And, even more than that, it’s the increased understanding of what the system actually needs to do that comes from those conversations.

Of course, you can still go on to automate the scenarios, but that’s a secondary benefit.

BDD goes deeper than I thought

As I’ve just explained, I realised belatedly that Behaviour-Driven Development is a framework and set of principles for thinking about the whole software lifecycle, and includes techniques to use to keep you focussed on those principles at all phases of a project, not just when writing test code.

But it turns out that I also wasn’t embracing BDD to the full when actually coding either! To see my problem, let’s take a look at the famous BDD “double loop” diagram. Here’s one version of it:

The behaviour-driven development “double loop”

Now, the Acceptance Tests that you write first are based on the examples you’ve already derived, which come out of the processes I talked about earlier (crucially, from conversations and shared understanding), so you should be able to trace the thread back from here to the business goals. I was doing the left-hand part of the loop right.

BDD and low-level testing

However, although BDD practitioners do talk about “Unit Tests”, to take BDD “all the way down”, I needed to start thinking less about unit-tests and more about low-level executable specifications (Acceptance Tests being the high-level executable specifications). This might sound like a simple difference in phrasing, but there’s more to it than that:

In BDD unit testing you focus on specifying (and thus testing) the behaviour of the class, rather than testing it method-by-method. This “outside-in” approach is a parallel of what you do with your automated acceptance tests at the higher level.

(Interestingly, this idea of outside-in unit testing has similarities with the so-called “London School” of TDD, but I’ll save the comparison of those for another time!)

As part of the shift away from unit tests and towards low-level executable specifications, you’ll want to write in something closer to the structured natural language used in your application-level BDD scenarios. There are plenty of tools around which allow you to do that at this lower level. Here’s an example from the PHPSpec manual:

<?phpnamespace spec;use PhpSpec\ObjectBehavior;

class MarkdownSpec extends ObjectBehavior{function it_converts_plain_text_to_html_paragraphs(){$this->toHtml("Hi, there")->shouldReturn("<p>Hi, there</p>");}}

PHPSpec, and other “xSpec” tools like it, can generate nice natural language output similar to that shown at the start of this article.

But you can do BDD at the low-level with “traditional” unit-testing tools too. Here are some principles from BDD in Action to help you do that:

• Write test methods names that describe the behaviour (using “should” helps here)
• Use test class names that provide context (using “when” helps here)

And here’s an example in PHPUnit — notice the “when” and “should”:

<?php

use PHPUnit\Framework\TestCase;

class WhenChangingUserStatus extends TestCase{/*** @test*/public function should_be_able_to_revert_status_changes(){// ...}}

Note the @test annotation since the method name itself doesn’t begin with test.

So, to do BDD all the way down, I needed to think in terms of outside-in testing and low-level executable specifications, not method-by-method unit tests.

There was one more area where I realised I could apply BDD more thoroughly — living documentation.

Living documentation can do more than I thought

The idea of living documentation, always up-to-date, is one of the things that’s always attracted me to BDD. However, I realised that I hadn’t seen the full potential of this idea.

My thinking around living documentation and BDD only went as far as encouraging people to share the feature files with the full project team. I hadn’t considered the possibility of integrating with project management processes or tools more closely, such as…

• Using BDD reporting tools (e.g. Serenity) to make the features easier to digest (providing summaries/aggregation and better navigation)
• Using the living documentation to demonstrate feature readiness (showing what can be deployed).
• Using the living documentation to show feature coverage (demonstrating progress and what has/hasn’t been built)

The first of these is fairly self-explanatory, but the second and third are a more radical departure from what I was originally doing, so let’s explore those in a little more depth.

Feature readiness

Looking at something as basic as a directory full of .feature files (or .story files in JBehave) can be a bit overwhelming, especially for a non-developer — where do you start, and how do you see the wood for the trees?

Making more use of BDD reporting tools, however, we could aggregate scenario results at the feature level, showing when a complete application feature was ready to deploy (i.e. all tests were passing).

Feature coverage

“Feature coverage” is an even more interesting idea — showing progress through a whole project by seeing how many acceptance criteria have been defined and automated for each requirement, including reporting on those which have no AC yet. To make the most of this, we’d need to combine a BDD reporting tool with our project-management tools, so that details of non-started stories could be retrieved.

This would be quite a major process and technical undertaking, however. In fact, almost all of what I’ve described so far has repercussions on process throughout a large part of our business, going way beyond development. So what am I, as a Tech Lead, going to do about all of this? That’s what I’ll explore in the final section.

So now what do I think?

As you’ve now seen, my research has highlighted a lot of areas where we’re not engaging with the full BDD methodology.

However, I’m not about to suggest that we overhaul all our processes to run projects “by the BDD book”. Before I jump in with both feet, I still have questions I want to explore. I’ll share some of those questions with you, and then finish with what I actually am going to do to keep these ideas moving forward nonetheless.

Questions raised for me

Done right, BDD helps you focus on business goals and drive towards shared understanding through conversations. However, BDD isn’t the only way to focus on business goals, and shared examples aren’t the only way to create shared understanding.

At White October, we’ve always prided ourselves on working “unusually closely” with our clients — we don’t have business analysts between developers and clients — clients are full team members from the word go, and work directly with developers and designers (among others). Do we really need new processes to help us adhere to the principles that BDD espouses or are our existing process doing that already?

As mentioned, we don’t have business analysts. As a small company, we don’t have QA staff either; the QA role is shared around different team members at different time. This means that a “3 Amigos” example-writing session — with Business Analyst, Tester and Developer — could end up being just one person! Of course, we’d bring in the client themselves rather than a BA, and a designer would be a natural replacement for the Tester role here — they have a different perspective which would really help. But would clients really be willing to commit to the level of involvement example-writing would require? This is quite different than running the meeting with solely in-house staff.

Part of the answer here is to say that uncovering examples via conversations is different to (and quicker than) writing the formal scenarios, but does that make it legitimate to write scenarios unilaterally?

We pride ourselves on doing Agile well, and that gives us strong emphasis on embracing uncertainty and change already. Do we need to adopt new techniques like Real Options to do this better?

As a developer, I like the idea of programatically tying together different tools, linking our project tracking software to outputs from our automated behavioural tests (as discussed in the living documentation section above). However, as a small company, do we need heavy processes to help us track progress better, or can we just rely on manual processes and conversations as at present?

What I’m doing

Whilst I do have questions around exactly what we should take from BDD, and whether a “pick-and-mix” approach is in fact the best for us, I’m keen that we keep moving forward in exploring BDD further. So here’s what I am doing:

• I’m going to think more about how I use language — are we doing enough to say “we do BDD”, or should I restrict myself to terms like “behavioural testing” and “executable specifications” which are more closely aligned with what we actually do?
• I’m going to lend the BDD in Action book to others in the business, particularly those involved in non-developer roles.
• I’ve also been thumbing through that book in meetings about how we do product strategy, showing key sections to our Product Strategists. And I’ve shared other related articles with the rest of the company too.
• I’ve recommended that my Project Manager and developer colleagues attend Cukenfest London, a BDD and agile conference.
• I’m going to keep on learning and reading myself, and suggesting smaller changes that bring us closer to the approaches BDD espouses, even if they’re lighter than the full techniques.

Although the questions in the previous section might make me sound like a skeptic, they’re more accurately seen as a reflection of the fact that I’m keen to engage seriously with BDD and what it means for us. I’m passionate about the principles of BDD and really pleased to understand them so much better now.

Conclusion

My research suggests that there are a lot of people out there like I used to be, who treat Behaviour-Driven Development as synonymous with automated behavioural testing. Automated behavioural testing is great, but hopefully this article has shown you that there’s a lot more to BDD than that, as it actually covers the full software development process.

Or if you already knew all that, then I’d love to hear your thoughts in the comments on the questions that this has raised for me. I’m looking forward to seeing how we take BDD forward, and maybe even sharing the next steps of our journey in a future blog post.