Why is a ‘Design-First’ Approach to Building APIs Controversial? How Does One Win at It?

‘Design-first’ is an approach to building APIs that requires an API definition to be committed before any code is written. In the arena of public opinion, it has been both booed and applauded.

In API product development, where the rubber meets the road, what obstacles do we meet along the way?

Let's take a dive deep into how to succeed in a design-first approach to APIs.

Front-Loading Design

Design-first requires an API definition, but what does that look like?

An API definition may come in a format readable by both humans and machines.

Which format to choose is often a secondary concern to the technology chosen for communication semantics.

Some examples:

• RESTful APIs may use OpenAPI Specification
• GraphQL APIs use the GraphQL Schema Definition Language
• gRPC APIs may use Protocol Buffers
• Event-based APIs may use AsyncAPI Specification

Below is an example of what an OpenAPI definition might look like for an API that helps consumers find famous quotes:

We start constructing an interface definition, a contract between API Producer and API Consumer. This serves as our guide to trekking further into the Software Development Lifecycle (SDLC). 📜

# quotable-openapi.yaml

openapi: 3.1.0
info:
  version: "1.0.0"
  title: Quotable API
servers:
  - url: https://quotable.apilab.io
paths:
  /quotes:
    get:
      operationId: listQuotes
      description: List all the famous quotes.
      responses:
        "200":
          description: OK
          content:
            "application/json":
              schema:
                $ref: "common-openapi.yaml#/components/schemas/QuoteList"
# Schemas are introduced in just a moment...

But how do we even get to this point? See 5 Developer Tips for Surviving API-First to get some ideas!

The Promise of Design-First

Many benefits have been promoted as guarantees on the box. Let's take a look at a few of them.

Reduced Development Costs

According to a frequently-cited research paper by IBM System Science Institute, fixing a bug in testing is 15 times more expensive than fixing it in design. Wowza.

Presumably, spending more time on design-first should reduce the number of bugs.

• Do these APIs support our acceptance criteria?
• Are we seeing good cohesion within each API?
• What are the coupling constraints?

Easy On-Boarding

API Consumers come in all shapes and sizes. They may be folks building applications that rely on an API. They may test engineers. They may even be new hires joining the API Producer team.

• Can we derive documentation from our API definition?
• Can we generate client SDKs?
• Can we scaffold assets such as API tests?

Early Design Governance

If we're brainstorming contracts early in the process, we have an opportunity to add a governance gate.

• Do all collection-based paths have plural names?
• Are parameters in camelCase?
• Do property names match our Ubiquitous Language?

# common-openapi.yaml

openapi: 3.1.0
info:
  version: "1.0.0"
  title: Common Components
paths: {}
components:
  schemas:
    QuoteList:
      type: array
      items:
        $ref: "#/components/schemas/QuoteItem"
    QuoteItem:
      type: object
      required:
        - id
        - content
        - author
        - authorId
        - authorSlug
        - length
      properties:
        id:
          type: string
        content:
          type: string
        author:
          type: string
        tags:
          type: array
          items:
            type: string
        authorId:
          type: string
        authorSlug:
          type: string
        length:
          type: number

This is all enabled by design-first. For example, schema definition becomes a point of review for APIs early in the cycle.

OpenAPI 3.1.0 introduced full support for JSON Schema 2020-12.

Are we good? May we pass?

The Downside of Design-First

While all of these benefits seem good and healthy, there are some historically detrimental anti-patterns re-surfacing to plague our efforts. Let's illuminate their disastrous intentions. 😈

Big Design Up Front

Ah, yes, our old nemesis, Big Design Up Front (BDUF). So insidious, it has a silly acronym. This anti-pattern comes to us from decades of following a waterfall model of SDLC.

The most common criticism of BDUF is its inability to pivot in the face of change. It eliminates a necessary feedback cycle between design and implementation, burning cycles without having the wisdom gleaned by concrete systems interactions.

Lava Flow

Team members move on to different projects. Trends in architecture change rapidly. What happens when design responsibility shifts over time? Knowledge of why decisions were made are often lost.

When we don't understand the context, we look to make changes with less information. This is a recipe for breaking changes and a pile of refactoring initiatives that may never complete.

This anti-pattern is known as Lava Flow. It's helpful to be aware of this at the onset of any API initiative.

Design by Committee

With no designated decision-maker, even a small design can take a disproportionately long time to nail down.

Coupled with Big Design Up Front, this is a significant contributing factor to project stagnation.

The Design by Committee anti-pattern is a nasty trap that leads to work that's behind schedule, over-budget, and fragmented.

Filling the Potholes

All is not lost. Fortunately for us, we have loads of experience in the industry and have several strategies from which we can learn.

Just Enough Design Initially

The inverse of BDUF is to allow all design to emerge from the implementation, sometimes referred to as Emergent Design, though definitions of this term may vary.

We need to be careful of reaching an over-reliance on allowing all design to emerge from code.

The danger is in spinning our precious SDLC cycles on code maintenance as we drift far from big picture guidance.

One compromising philosophy is known as Just Enough Design Initially (JEDI).

"However, my basic rule of thumb for knowing when enough modeling has been done up front is, when after one pass through the envisioned scope of the software in question, modeling in small groups does not produce any new classes or associations of real significance." - Steve Palmer, Feature Driven Development, 2003

We do spend time on design-first, but we keep it within the boundaries of what we understand in the moment, allowing more data to influence our design as we begin a technical implementation.

Iterative Design

If we spend just enough time at the start of the API lifecycle thinking about design, what steps can we take to create a tight feedback loop?

For APIs, this means:

1. Start with a design document, such as an OpenAPI definition, using tools that enable collaboration.
2. Launch a mock server based on this design.
3. Write both API Producer and API Consumer contract tests to inform the next iteration.

Optionally, as the design iterations continue, techniques like dynamic routing via an API Gateway can direct clients to either the mock server or a prototype of the API server as it's built.

This strategy helps us secure a design while putting in just enough implementation to prove our design actually works in the real world! 🎉

API as a Product

When we think about software products, there's typically a human-facing interface, such as a web application. Our APIs are more than just integration vehicles. They require the care necessary for any product in our catalogue.

Here are some tips to help legitimize an API as a product:

• Design is always prioritized.
• Appoint an API Product Manager to help guide the experience, stakeholder communication, and development.
• Plan for longevity. The contracts we create between API Producer and Consumer need care and documentation.
• Consider Architectural Decision Records to document the motivating factors and evaluated solutions, helping future owners understand how we got here.

Further reading on how teams may evolve to meet our API product needs: Scale API teams with Platform Ops.

Zooming Forward

At a high level, many of these concepts can be applied to all software development. Here are some of the benefits:

• API implementations will have high levels of cohesion.
• There will be less rework post-implementation.
• The design and implementation are more adaptive to change.

Design-first may require a big shift in how we think about our API lifecycle. Keep all of this in mind, and our future selves will thank us. 😌

Looking for a place to chat more about design-first with OpenAPI? Check out the OpenAPI Discord community!

Social photo by Rick J. Brown on Unsplash

Co-published here.