Developers Should Drive Their E2E Tests to Cypress

All software companies strive to produce quality products. The journey begins with a product that works. Software correctness is usually defined as the adherence of software to its specification. In most cases, formal verification is not possible. As such, we typically impose functional correctness with test coverage.

To rise to the challenge, companies cultivate a culture that encourages developers of all stacks to write tests of varying scope. This culture usually gives birth to an abundance of static, unit, and integration tests written by developers. However, many companies do not offer their developers the tools to write end-to-end (E2E) tests.

In a world of distributed systems of growing complexity, is that enough?

Motivation

Isolated component tests are likely to fail to spot critical bugs. We’ve all faced the harsh reality of a seemingly perfectly covered new feature crashing and burning over a pesky inter-service bug.

Wouldn’t the world be a better place if developers could write a few E2E tests to verify the users get the experience we want to deliver? In many companies, these tests are the domain of software automation engineers. Traditionally, they're written on dedicated frameworks such as Selenium.

As such, they're largely unfamiliar to the developers. For front-end engineers, they are sometimes written in a different programming language. This artificial boundary restricts developers from making the most out of automated testing.

The advantages of developer-driven E2E testing are clear:

Higher product quality and maintainability — Supplementing the developers' toolbox with E2E tests encourages and improves the adoption of Test-Driven Development, with its numerous benefits. Having these tests as part of the main task rather than an appendix assures the feature is truly covered before release.

An atmosphere of ownership — Owning the feature’s entire lifecycle fosters responsibility for the quality of the finished product and boosts the developers’ perceived impact. Reliance on external testing (whether automated or manual) reduces developer accountability. This poses the risk of a shallow sense of ownership and a “not my problem” mentality.

Faster development — Endorsing a holistic approach to testing by assigning the entire testing pyramid or trophy to developers hastens development. Firstly, it removes the overhead of involving an additional party in the development process. Secondly, it prevents pipeline-lengthening test redundancy. After all, there is no need to burden the build pipeline with an E2E test where an existing lean integration test suffices.

Operating cost reduction — To add to the previous point, fewer tests (especially hefty Selenium suites) demand fewer resources to execute.

The Solution

Make E2E tests a common commodity. Empower developers with light and simple tools and methodologies to start testing their products.

The Road to Success

What tools? I won’t repeat the many trade-offs that come with any of the existing E2E testing frameworks. The subject is out of the scope of this story and has been covered extensively (e.g.Selenium vs. Puppeteer and TestCafe vs. Cypress). Instead, I’ll share the gist of my process:

After working with Selenium, I felt it was not a suitable solution. The bulkiness of the WebDriver and its limited mocking capabilities render it impractical for TDD, and the framework’s learning curve was too steep for a ubiquitous instrument.

In the same corner lies the newer Puppeteer framework, with which I had experience in browser automation. Unfortunately, along with the recently-released Playwright, it suffers many of the same pain points.

The TestCafe framework, while ticking off many boxes, uses a test paradigm that may not feel natural for front-end developers who are used to the common testing frameworks.

Some of the gaps in these frameworks can be filled by CodeceptJS, which is a versatile wrapper. Yet, its limited popularity makes it a tougher commitment in an enterprise setting

Eventually, I settled on a POC for Cypress.

Why Cypress? In their own words:

Our mission is to build a thriving, open source ecosystem that enhances productivity, makes testing an enjoyable experience, and generates developer happiness. We hold ourselves accountable to champion a testing process that actually works.

First and foremost, I felt Cypress is a choice for developer happiness. After all, enjoyability makes or breaks the adoption of new tools and ideas. If we are to reap the fruits of developer-driven E2E testing, developers should want to write tests.

Secondly, I saw great potential in Cypress’s extensibility to facilitate some of my product’s unique testing requirements. Thirdly, its popularity and community support make it a safer choice for the future.

With the proper wrapping and extension, Cypress helps developers integrate testing into their everyday workflow. As developers code, a simple terminal command can launch an impressive test runner that executes tests on a real browser.

The same test runner, in the same run environment, can then be used in the CI pipeline of your choice. The results are presented in beautiful reports that may include screenshots, recordings of test runs, and anything else you’d like to attach.

When it comes down to it, what does Cypress offer?

Blazing speed — It’s quick. Quick enough to test our code continuously. Upfront, this is due to Cypress being the only framework mentioned that runs in the browser. However, one of its chief performance boosts is derived from the built-in and extended back-end mocking capabilities.

In E2E tests, why waste precious seconds waiting for buttons to be clickable when you can replay a recorded page startup? In more limited test scopes, why should you wait for the server to execute a complicated query when an appropriate JSON response suffices?

Easy to adopt and use — New tools always involve a learning curve. Out of the box, Cypress utilizes the ubiquitous Mocha, Chai, and Sinon test stack. If that isn’t your cup of tea, you might enjoy some of the exciting plug-in support, including Jest-style assertions, React Testing Library, and even Gherkin syntax.

Holistic approach — Cypress is not limited to E2E testing. It can support tests of all scopes. Beyond the innate scoping determined by modifying the extent of back-end mocking, there are specialized plug-ins for component testing.

Debugging simplicity — Readable error messages and stack traces, access to developer tools, test recording and snapshotting, all mount up to a simple debugging experience that is less likely to leave you wondering why your test failed.

Integration ready — Developers can run the same tests they used during development right in the CI pipeline. Cypress offers examples to most providers. Official Docker images are available to run Cypress locally and in CI.

Extensibility — Cypress is a popular project with enterprise backing and significant community support. The ecosystem already includes many plug-ins ranging from authentication to framework extensions. This extensibility extends to you — it’s relatively easy to write your plug-in with the existing API. Little stops you from fitting Cypress to your needs.

To successfully integrate Cypress into the workflow, one might need to undergo a process of customization to make it as accessible as possible in the context of their product and development procedures.

Accessibility may translate to minimal setup, comfortable and familiar configuration, ready-to-use commands, support for the preferred test stack syntax, performance tuning, and CI integration.

Conclusion

As I see it, empowering software developers with E2E testing can have a far-reaching impact on R&D and organizations as a whole.

It’s important to note that E2E tests don’t replace component test coverage, but unlock a holistic testing scheme. This approach drives higher product quality and maintainability, promotes an atmosphere of ownership, leads to faster development, and reduces operating costs.

For instance, developers working on a component should complement the unit-tests with an E2E test that checks its functionality from a user’s standpoint.

Towards that end, I find that Cypress is geared towards developer-driven testing. Backed by blazing speed and ease of use, it has the potential to keep developers happy and motivated to adopt this approach. Coupled with Cypress’s impressive extensibility, anyone is free to fit it to their needs.

I’d like to stress that E2E developer-driven testing does not necessarily come at the expense of automation by external parties (e.g., software automation engineers). Testing by external parties has its benefits, mainly rooted in the desire to avoid letting the fox guard the henhouse.