The 100% Code Coverage Myth

(Licensed from Adobe Stock Photo) There’s a lot of advice around the internet right now saying that 100% coverage is not a worthwhile goal. I strongly disagree. Usually, code being hard to test is a sign that is needs to be refactored. I get it. A few years ago, I sucked at testing. I thought it was just something that would make me move more slowly. It simply wasn’t a thing that people did very often when I started coding. If it was, it was often a separate QA team that was responsible. A few years back though it became a real hot topic. Interviews started expecting candidates to know how to write tests, and more organizations were pushing it from the top down as a quality initiative. I always strive to be at the top of my game, and I decided walking into interviews and saying “testing isn’t really my strong suit” was no longer a good look, so I decided I was going to get 100% coverage on all of my tests from then on. At the time, I wasn’t really sure what benefits I’d get out of it, or if there really were any. Now, I wouldn’t go back. When something breaks in a code base with 100% coverage, it is very likely your tests will tell you exactly where and how. honestly This isn’t to say unit testing is all you need. It isn’t. But leaving code untested is not a good option in my opinion either. Come back with me, to a time when I didn’t believe in the benefits of test coverage either. Part 1: Learning the Lingo At the time, the tools of the trade were a combination of , , and . Mocha was the test-runner, sinon provided the ability to create “mocks” and “spies”, and chai is an assertion library, so you can type assertions in a human language friendly manner. mocha sinon chai (Is this a spy?) I basically had no idea what any of this meant. Before I could be effective, the first thing to do was learn the language. So, first thing’s first — the hell is a spy or a mock? Although the first thing that comes to mind is James Bond or Ethan Hunt. That is definitely not what we are talking about here, though it isn’t a terrible metaphor. After reading some documentation I eventually learned that a spy is a function that has been modified by a testing framework to provide meta information about how it has been used. It spies on it. Kinda like how people could spy on you with Apple’s recent FaceTime Bug. So kinda like James Bond. A mock is similar to a spy but it has been modified even more. As well as providing and keeping track of how a particular function has been used, it also changes its behavior to be predictable. I also learned there are several types of testing. Not limited to the three most common: Unit Testing, Integration Testing, and E2E Testing. When we are “unit testing” that means we need to be able to break down our code into individual units. Anything outside of that particular unit is a candidate to be mocked, such as other functions or entire modules. Jest is my tool of choice for unit testing. Unit testing is the only type of testing where coverage is measured. When we are Integration Testing, we are testing the integration of our software with other pieces of software, such as a test that passes a message through Kafka that our service should receive, and that the result of that can be found in the database afterward. I also usually reach for Jest when creating Integration tests. E2E Testing is kinda like a bot using your app. You program it to load the site in a browser, click things, and ensure everything works as expected from a user’s perspective. Cypress is my favorite tool on this area, but that didn’t exist back when I was learning. Selenium was the big player of the day, and to be honest, it was a big enough domain I was happy to let a QA Automation Engineer handle that part. With new knowledge in hand now came the hard part: putting it to practice. I spent several months making sure every single piece of code I wrote had test coverage. At first, I admit, it was quite difficult. I spent a lot of time on StackOverflow looking up mocking and spying examples. By the end I found that I the amount of confidence I had in my code was substantially higher. Another benefit was when something broke my tests would usually tell me exactly where. When other engineers made changes to code that I made I could review it much more quickly. When important APIs changed, people were alerted via a failing test and either quickly updated it or gave their changes a second thought. More than that, I started writing better code. I learned that usually if something is hard to test, or hard to fully cover, it usually meant I didn’t write that code very well, and it could be refactored resulting in more maintainable and flexible APIs. To that end, trying to reach 100% coverage encouraged me to extract anonymous functions into named functions, and to understand partial application and dependency injection in many refactors. After getting integrations tests down as well, I even gave up GitFlow for trunk-based development. Committing to master was something I thought was crazy a few years back, and now I do it on a team of nearly 15 engineers every day. : Related Read I Have a Confession to Make. I Commit to Master Part 2: Lead by Example Around the time I was getting pretty confident with my new testing stack, another tool was introduced to the market which many claimed made unit testing even simpler: Jest. Jest is an automated testing framework pioneered by Facebook. Jest does a really awesome job condensing the previous libraries I had used into a single coherent framework that is a test-runner, as well as a set of APIs for mocking, spying, and assertions. Beyond providing a single library with all your unit-testing needs, Jest does a great job at simplifying some of the concepts and patterns as well with powerful and simple mocking. Because I think Jest is simpler to use and to understand, I’m going to stick with Jest for examples. If you’re just joining me on this article, that’s fine —what you’ve read so far is meant to stand on it’s own. However, I’ve been documenting the process of building a React application using Parcel with Streaming SSR, and this article is going to continue where the last part left off. In my last article, linked below, I showed how to set up Jest with code coverage and said in the next article I’d show how to get the coverage up to 100%. : Related Read Enforcing Code Quality for Node.js I figured the best way to demonstrate 100% coverage is showing how to get there. Throughout the journey we will likely discover several places where code can be refactored to be more testable. So, I’ll continue where I left off, and get coverage of this project to 100%, and show what refactors to make, where to use partial application and dependency injection, and what to mock along the way when coverage is difficult to get. So… Let’s get started. Here’s the project I’ll be working on: : GitHub Link patrickleet/streaming-ssr-react-styled-components The project has a react app in the app folder, and a server folder which contains the SSR logic. Let’s start with the application tests. Application Tests In , after configuring Jest, I got started with a simple test for a simple component. I have several React components that are equally as simple. the last article This is one of the reasons that functional components are really powerful. Functions are easier to test than classes. They don’t have state — instead they have inputs and outputs. Given input X, they have output Y. When there is state it can be stored externally to the component. The new React Hooks API is nice in this regard because it encourages making functional components, and has an easily mockable mechanism to provide state to the component. Redux provides the same benefit in regards to testing. Let’s start by knocking out the rest of the simple components. We basically just need to render them and maybe check that some important pieces of info are rendered. I usually put code inline in the articles, but there’s not really anything new in these tests, so instead I’ve decided to link to the actual commits and only show one full example: Let’s take a look at the About page: React Helmet Page About = ( About Page This is the about page ) About import from 'react' import from 'react-helmet-async' import from '../components/Page' const => () Page Page export default And it’s tests: React { shallow } About describe( , () => { it( , () => { expect(About).toBeDefined() tree = shallow( import from 'react' import from 'enzyme' import from 'app/pages/About.jsx' 'app/pages/About.jsx' 'renders About page' const ) expect(tree.find('Page')).toBeDefined() expect( tree .find('Helmet') .find('title') .text() ).toEqual('About Page') expect(tree.find('div').text()).toEqual('This is the about page') }) }) About All of the tests in the following commits are very similar: fix: tests for pages fix: tests for components test: style component renders test As you can see, just making sure our component renders is enough for these components to get 100% coverage. More detailed interactions are better left to E2E tests, which is out of scope for the current article. The next component, is slightly more complex. After writing a rendering test, you’ll notice there is still an unreachable anonymous function that is used in the Router to render the About page. app/App.jsx In order to access and test this, we want to make a small refactor, extracting the function to a named function so we can export it and test it out. Now it is easy to test: Because we have another set of tests for the About page above, we’ll leave its more specific tests to live there, and just need to check that it renders here. And with that, the only file left to test in our application is , and then we can move on to finishing up server side tests. app/client.js Let’s take a look at the code: React ReactDOM { HelmetProvider } { BrowserRouter } { rehydrateMarks } importedComponents App element = .getElementById( ) app = ( ) (process.env.NODE_ENV === ) { rehydrateMarks().then( { ReactDOM.hydrate(app, element) }) } { ReactDOM.render(app, element) } ( .hot) { .hot.accept() } import from 'react' import from 'react-dom' import from 'react-helmet-async' import from 'react-router-dom' import from 'react-imported-component' import from './imported' // eslint-disable-line import from './App' const document 'app' const HelmetProvider HelmetProvider // In production, we want to hydrate instead of render // because of the server-rendering if 'production' // rehydrate the bundle marks => () else // Enable Hot Module Reloading if module module The first thing I notice is that there is a reliance on global variables — , and . The second thing is that nothing is exported so it may be hard to run multiple times with different inputs. document process module We can remedy this with a few refactors: Wrap up all of the logic into a function that we can export. This function will accept an options objects with all of its dependencies. This is called . This will allow us to easily pass along mock versions of a bunch of things if we so choose. dependency injection We have an anonymous function in production mode after rehydrating which should be extracted to a named function. We also will want to mock a few of the external modules: , , and . Modules are a form of dependency injection themselves. react-dom react-imported-component app/imported.js First here’s the newly refactored file with the changes in bold: React ReactDOM { HelmetProvider } { BrowserRouter } { rehydrateMarks } importedComponents App hydrate = () => { ReactDOM.hydrate(app, element) } start = ({ isProduction, , , hydrate }) => { element = .getElementById( ) app = ( ) (isProduction) { rehydrateMarks().then(hydrate(app, element)) } { ReactDOM.render(app, element) } ( .hot) { .hot.accept() } } options = { : process.env.NODE_ENV === , : , : , hydrate } start(options) import from 'react' import from 'react-dom' import from 'react-helmet-async' import from 'react-router-dom' import from 'react-imported-component' import from './imported' // eslint-disable-line import from './App' // use "partial application" to make this easy to test export const ( ) => app, element export const document module const document 'app' const HelmetProvider HelmetProvider // In production, we want to hydrate instead of render // because of the server-rendering if // rehydrate the bundle marks from imported-components, // then rehydrate the react app else // Enable Hot Module Reloading if module module const isProduction 'production' document document module module Now we can actually access and test start with a variety of options as well as testing hydrate independently of the startup logic. The tests are a bit long, so I’ve put comments inline to explain what is going on. Here are tests for the file: React fs path { start, hydrate } { JSDOM } jest.mock( ) jest.mock( ) jest.mock( ) pathToIndex = path.join(process.cwd(), , ) indexHTML = fs.readFileSync(pathToIndex).toString() DOM = JSDOM(indexHTML) = DOM.window.document describe( , () => { it( , () => { element = .getElementById( ) expect(element.id).toBe( ) }) }) describe( , () => { afterEach( { jest.clearAllMocks() }) describe( , () => { it( , () => { ReactDOM = ( ) = { : { : jest.fn() } } options = { : , , } start(options) expect(ReactDOM.render).toBeCalled() expect( .hot.accept).toBeCalled() }) it( , () => { ReactDOM = ( ) importedComponent = ( ) importedComponent.rehydrateMarks.mockImplementation( .resolve()) = {} hydrate = jest.fn() options = { : , , , hydrate } start(options) expect(ReactDOM.render).not.toBeCalled() expect(hydrate).toBeCalled() }) }) describe( , () => { it( , () => { ReactDOM = ( ) element = .getElementById( ) app = ( ) doHydrate = hydrate(app, element) expect( doHydrate).toBe( ) doHydrate() expect(ReactDOM.hydrate).toBeCalledWith(app, element) }) }) }) import from 'react' import from 'fs' import from 'path' import from 'app/client' import from "jsdom" 'react-dom' 'react-imported-component' 'app/imported.js' // mock DOM with actual index.html contents const 'app' 'index.html' const const new const document // this doesn't contribute to coverage, but we // should know if it changes as it would // cause our app to break 'app/index.html' 'has element with id "app"' const document 'app' 'app' 'app/client.js' // Reset counts of mock calls after each test => () '#start' 'renders when in development and accepts hot module reloads' // this is mocked above, so require gets the mock version // so we can see if its functions are called const require 'react-dom' // mock module.hot const module hot accept // mock options const isProduction false module document module 'hydrates when in production does not accept hot module reloads' const require 'react-dom' const require 'react-imported-component' => () Promise // mock module.hot const module // mock rehydrate function const // mock options const isProduction true module document '#hydrate' 'uses ReactDOM to hydrate given element with an app' const require 'react-dom' const document 'app' const div div const typeof 'function' Now when we run our tests, we should have 100% coverage of the folder, aside from which is a generated file, and doesn’t make sense to test as it could generate differently in future version. app app/imported.js Let’s update our jest config to ignore it from coverage statistics, and check out the results. In add: jest.config : [ , ] "coveragePathIgnorePatterns" " /app/imported.js" "/node_modules/" Now when we run we get the following results. npm run test : GitHub Link test: client.js tests · patrickleet/streaming-ssr-react-styled-components@c5fcfe9 Something that I want to point out, is that while I’m developing tests, I’m usually using “watch” mode to do so, so as tests are changed they are automatically re-run. With application tests done, let’s move on to the server. Server Tests In the previous article I wrote tests for one application file, as well as one server file, so we for . Now we need to test the three remaining files in . already have tests server/index.js server/lib Let’s start with : server/lib/client.js fs path cheerio htmlPath = path.join(process.cwd(), , , ) rawHTML = fs.readFileSync(htmlPath).toString() parseRawHTMLForData = { $template = cheerio.load(template) src = $template(selector).attr( ) { src } } clientData = parseRawHTMLForData(rawHTML) appString = splitter = [startingRawHTMLFragment, endingRawHTMLFragment] = rawHTML .replace(appString, ) .split(splitter) getHTMLFragments = { startingHTMLFragment = [startingHTMLFragment, endingRawHTMLFragment] } import from 'fs' import from 'path' import from 'cheerio' export const 'dist' 'client' 'index.html' export const export const ( ) => template, selector = '#js-entrypoint' const let 'src' return const const ' ' const '###SPLIT###' const ` ` ${appString} ${splitter} export const ( ) => { drainHydrateMarks } const ` ` ${startingRawHTMLFragment} ${drainHydrateMarks} return First off, I’ve noticed there’s a pretty big block of code that isn’t even used in the project from a previous abandoned strategy. Everything from through . export const parseRawHTMLForData const clientData I’m gonna start by deleting that. The less code there is, the less places bugs can exist. There’s also a couple of exports which I never made use of which can stay private to the module. Here’s the updated file: fs path htmlPath = path.join(process.cwd(), , , ) rawHTML = fs.readFileSync(htmlPath).toString() appString = splitter = [startingRawHTMLFragment, endingRawHTMLFragment] = rawHTML .replace(appString, ) .split(splitter) getHTMLFragments = { startingHTMLFragment = [startingHTMLFragment, endingRawHTMLFragment] } import from 'fs' import from 'path' const 'dist' 'client' 'index.html' const const ' ' const '###SPLIT###' const ` ` ${appString} ${splitter} export const ( ) => { drainHydrateMarks } const ` ` ${startingRawHTMLFragment} ${drainHydrateMarks} return It looks like one test should probably do it for this one. However, there’s a slight hiccup in the plan: this file depends on the build being run before as it reads in the generated build. Technically this makes sense, because you’d never try to render the app on the server without having a built app to render. Given that constraint I’d say it’s ok, and probably isn’t worth the effort to refactor given we can just make sure our pipeline calls build before test. If we wanted to have really pure unit isolation we might consider refactoring a bit more as technically the whole application is a dependency of SSR, so it could be mocked. On the other hand, using the actual build is probably more useful anyway. You’ll frequently encounter trade-offs like this throughout the process of writing tests. With that being said, here is the test to get full coverage for this module: { getHTMLFragments } describe( , () => { it( , () => { drainHydrateMarks = [start, end] = getHTMLFragments({ drainHydrateMarks }) expect(start).toContain( ) expect(start).toContain(drainHydrateMarks) expect(end).toContain( ) }) }) import from 'server/lib/client.js' 'client' 'exists' const ' ' const ' ' 'script id="js-entrypoint"' And the commits: , . fix: remove unused code for parsing template test: server/lib/client tests Next, is quite tiny, so let’s knock that one out. Here is its code to refresh your memory, or if you’re just joining us now: server/lib/server.js express server = express() serveStatic = express.static import from 'express' export const export const And the tests: express { server, serveStatic } describe( , () => { it( , () => { expect(server).toBeDefined() expect(server.use).toBeDefined() expect(server.get).toBeDefined() expect(server.listen).toBeDefined() expect(serveStatic).toEqual(express.static) }) }) import from 'express' import from 'server/lib/server.js' 'server/lib/server' 'should provide server APIs to use' Seems how we are basically just deferring all the responsibility to express, and we expect express to provide this contract, we can just simply make sure it does, and it doesn’t really make sense to go beyond this. Finally, we have only one more file to test: . server/lib/ssr.js Here’s our module: ssr React { renderToNodeStream } { HelmetProvider } { StaticRouter } { ServerStyleSheet } { printDrainHydrateMarks } log through App { getHTMLFragments } (req, res) => { context = {} helmetContext = {} app = ( ) { sheet = ServerStyleSheet() stream = sheet.interleaveWithNodeStream( renderToNodeStream(sheet.collectStyles(app)) ) (context.url) { res.redirect( , context.url) } { [startingHTMLFragment, endingHTMLFragment] = getHTMLFragments({ : printDrainHydrateMarks() }) res.status( ) res.write(startingHTMLFragment) stream .pipe( through( { .queue(data) }, { .queue(endingHTMLFragment) .queue( ) } ) ) .pipe(res) } } (e) { log.error(e) res.status( ) res.end() } } import from 'react' import from 'react-dom/server' import from 'react-helmet-async' import from 'react-router-dom' import from 'styled-components' import from 'react-imported-component' import from 'llog' import from 'through' import from '../../app/App' import from './client' // import { getDataFromTree } from 'react-apollo'; export default const const const HelmetProvider context {helmetContext} HelmetProvider try // If you were using Apollo, you could fetch data with this // await getDataFromTree(app); const new const if 301 else const drainHydrateMarks 200 ( ) function write data this ( ) function end this this null catch 500 It’s a bit long, and there are a few paths to execute. I do want to make a couple small refactors that will make isolation a bit easier, such as extracting the logic to generate the app out to a separate function, and using partial application to be able to inject the application stream renderer so we can easily mock some redirects. Also write and end are a bit tough to get to, so we can pull those out higher using partial application as well. Here’s an updated version: React { renderToNodeStream } { HelmetProvider } { StaticRouter } { ServerStyleSheet } { printDrainHydrateMarks } log through App { getHTMLFragments } getApplicationStream = { helmetContext = {} app = ( ) sheet = ServerStyleSheet() sheet.interleaveWithNodeStream( renderToNodeStream(sheet.collectStyles(app)) ) } { .queue(data) } end = { .queue(endingHTMLFragment) .queue( ) } ssr = (req, res) => { { context = {} stream = getApplicationStream(req.originalUrl, context) (context.url) { res.redirect( , context.url) } [startingHTMLFragment, endingHTMLFragment] = getHTMLFragments({ : printDrainHydrateMarks() }) res.status( ) res.write(startingHTMLFragment) stream.pipe(through(write, end(endingHTMLFragment))).pipe(res) } (e) { log.error(e) res.status( ) res.end() } } defaultSSR = ssr(getApplicationStream) defaultSSR import from 'react' import from 'react-dom/server' import from 'react-helmet-async' import from 'react-router-dom' import from 'styled-components' import from 'react-imported-component' import from 'llog' import from 'through' import from '../../app/App' import from './client' // import { getDataFromTree } from 'react-apollo'; const ( ) => originalUrl, context const const HelmetProvider context {helmetContext} HelmetProvider const new return export ( ) function write data this // partial application with ES6 is quite succinct // it just means a function which returns another function // which has access to values from a closure export const => endingHTMLFragment ( ) function end this this null export const => getApplicationStream try // If you were using Apollo, you could fetch data with this // await getDataFromTree(app); const const if return 301 const drainHydrateMarks 200 catch 500 const export default Here’s a link to look at the diffs in Github: , and . chore: refactor ssr to break it up / make it easier to read chore: refactor ssr more Now let’s write some tests. We’ll need to set the jest-environment for this file specifically for node otherwise the styled-components portion will not work. defaultSSR, { ssr, write, end } jest.mock( ) mockReq = { : } mockRes = { : jest.fn(), : jest.fn(), : jest.fn(), : jest.fn(), : jest.fn(), : jest.fn(), : jest.fn() } describe( , () => { describe( , () => { it( , () => { req = .assign({}, mockReq) res = .assign({}, mockRes) getApplicationStream = jest.fn( { context.url = }) doSSR = ssr(getApplicationStream) expect( doSSR).toBe( ) doSSR(req, res) expect(res.redirect).toBeCalledWith( , ) }) it( , () => { log = ( ) req = .assign({}, mockReq) res = .assign({}, mockRes) getApplicationStream = jest.fn( { ( ) }) doSSR = ssr(getApplicationStream) expect( doSSR).toBe( ) doSSR(req, res) expect(log.error).toBeCalledWith( ( )) expect(res.status).toBeCalledWith( ) expect(res.end).toBeCalled() }) }) describe( , () => { it( , () => { req = .assign({}, mockReq) res = .assign({}, mockRes) defaultSSR(req, res) expect(res.status).toBeCalledWith( ) expect(res.write.mock.calls[ ][ ]).toContain( ) expect(res.write.mock.calls[ ][ ]).toContain( ) }) }) describe( , () => { it( , () => { context = { : jest.fn() } buffer = Buffer.from( ) write.call(context, buffer) expect(context.queue).toBeCalledWith(buffer) }) }) describe( , () => { it( , () => { context = { : jest.fn() } endingFragment = doEnd = end(endingFragment) doEnd.call(context) expect(context.queue).toBeCalledWith(endingFragment) expect(context.queue).toBeCalledWith( ) }) }) }) /** * @jest-environment node */ import from 'server/lib/ssr.js' 'llog' const originalUrl '/' const redirect status end write on removeListener emit 'server/lib/ssr.js' 'ssr' 'redirects when context.url is set' const Object const Object const ( ) => originalUrl, context '/redirect' const typeof 'function' 301 '/redirect' 'catches error and logs before returning 500' const require 'llog' const Object const Object const ( ) => originalUrl, context throw new Error 'test' const typeof 'function' Error 'test' 500 'defaultSSR' 'renders app with default SSR' const Object const Object 200 0 0 ' ' 0 0 'window.___REACT_DEFERRED_COMPONENT_MARKS' '#write' 'write queues data' const queue const new 'hello' '#end' 'end queues endingFragment and then null to end stream' const queue const ' ' const null As this file was a bit more complex than some of the others it took a few more tests to hit all of the branches. Each function is wrapped in its own describe block for clarity. Here is the commit on Github: . test: ssr unit tests Now, when we run our tests we have 100% coverage! Finally, before wrapping things up, I’m going to make a small change to my jest.config to enforce 100% coverage. Maintaining coverage is much easier than getting to it the first time. Many of the modules we tested will hardly ever change. : { : { : , : , : , : } }, "coverageThreshold" "global" "branches" 100 "functions" 100 "lines" 100 "statements" 100 And done! Here’s the commit on Github: . chore: require 100% coverage Conclusion My goal for this article was to demonstrate the techniques needed to be able to refactor your code, or isolate units using mocks and dependency injection to make tough to test code easy to reach and discuss some of the merits of reaching 100% coverage. Also, using TDD from a starting point is a lot easier. I’m a firm believer that if 100% coverage is hard to reach it’s because code needs to be refactored. In many cases an E2E test is going to be a better test for certain things. A suite on top of this which loads the app and clicks around would go a long way in increasing our confidence even further. Cypress.io I believe working in a codebase that has 100% coverage does a great job in increasing the confidence you have in each release and therefore increasing the velocity which you can make and detect breaking changes. As always, if you’ve found this useful, please leave some claps, follow me, leave a star on , and/or share on social networks! the GitHub project In the next part, coming soon, we will add a production ready Dockerfile, and explore how using nothing but another Dockerfile we can alternatively package our application as a static site served with Nginx, and some tradeoffs between the two approaches. Best, Patrick Lee Scott Check out the other articles in this series! This was Part 4. Part 1: Move over Next.js and Webpack 🤯 Part 2: A Better Way to Develop Node.js with Docker Part 3: Enforcing Code Quality for Node.js Part 5: A Tale of Two (Docker Multi-Stage Build) Layers Part 6: Bring in the bots, and let them maintain our code