Engineering Take Home Tests Done Right

Engineering take-home tests are a controversial thing. While some employers swear by them, just as many engineers believe they are disrespectful, lazy, and flat out refuse to proceed with interviews that include them.

The truth is: Take-home interviews are awesome, but they’re really easy for startups to get disastrously wrong.

A good take-home offers several advantages over other testing methods:

• It provides a more accurate work sample than whiteboard interviews or timed challenges.
• It removes the stress that unrealistically high-pressure tests — like timed challenges — induce.
• It creates easy benchmarks because every engineer should receive the same test.

Unfortunately, hiring managers often give out take-home projects that take days to complete, have no relation to the work an engineer will be doing at the company, and stress a potential engineer more than an algorithmic interview would.

To create a take-home that engineers will actually complete, while still giving you a good measure of their abilities, follow these three principles closely:

1. Respect Engineers’ Time

The more time your take-home requires, the less likely it is that engineers will complete it.

Most engineers interested in your role will also be talking to other companies, many of which will require take-home projects. Take-home projects stack up, and assuming the engineers are still working a full-time job, they simply won’t have time to complete a four-hour project for free. Additionally, good engineers know how much their time is worth — asking for four hours of free code is going to lead experienced engineers to dismiss your test.

If you really need to assign a project that takes longer than four hours, you should pay candidates for their code. If you don’t have the resources to do so, assign a shorter take-home.

To make sure your take-homes don’t eat up days of engineering time, scope the project down to a few features, and, unless you are hiring specifically for expertise in a niche technology, do your best to make sure the project doesn’t require familiarity with an obscure framework that will cost engineers hours of research time.

Finally, set clear expectations around how long the work should take. If it should only take two hours, make that clear in your instructions.

Audio streaming service TuneIn is one example of a startup that clearly time-boxes its take-home challenge, which you can view in this GitHub repository. The instructions being with:

Hey Front End developer. Welcome. Your mission, should you choose to accept it, is to carve out 2 hours and create a single page app (SPA) using the mini TuneIn station API defined below.

Hey this is important! We hope you can spend about two hours on this project. If you can finish faster — great! If not, limit yourself and don’t spend much longer than 2 hours MAX.

TuneIn’s team also went out of their way to limit setup time. Boilerplate code is provided (though candidates are free to discard it, if they prefer), and they share a simple API from which candidates can call data.

As a result, candidates can get up and running quickly and focus on solving the actual challenge posed by the take-home. They also have a clear expectations of time. If the project takes more than two hours to complete, it’s time to stop.

How companies get this wrong:

• Assigning projects at a scale that would take even senior engineers days to complete
• Assigning projects that require in-depth knowledge of something so niche most engineers would need to spend hours researching it
• Assigning projects that require significant setup time before a candidate can begin working

2. Make Your Test Close To A Work Product

The major advantage of a take-home project, from a hiring perspective, is that you get to see what it would be like if candidates worked on your production code. The closer their project is to the real work you do, the better signal you’ll get for how they’ll contribute to your team.

To do this well, construct a project that simulates key features of your product without replicating its full complexity. Workyard, a company that connects construction companies with workers, does this in its front-end take-home. The team explains their mission like this:

The mission, should you choose to accept, is to build a “Post a Project” modal in React. We’ll POST your new projects to Workyard’s staging API. After the project is successfully posted, we’ll show all your new projects in a list of very simple cards.

They include Sketch files mocking up the modal for candidates, so engineers aren’t left trying to design a front-end on their own:

Along with these images, Workyard also provides in-depth documentation of their team’s coding conventions. (Below is just a snapshot. The actual list is much longer.):

- Skinny components, fat helpers. Put as much complexity as possible in to helper files.

- Make mapStateToProps full of helper calls, leave render() as skinny as possible.

Their front-end project asks engineers to create a functional piece of code that mimics an actual feature of Workyard — posting projects — and just as importantly, it asks candidates to do it following Workyard’s conventions. Candidates are given technical specs in detail, just as they would on an actual product. They are also given a phone number and other contact information to ask the engineering team questions.

By setting the project up this way, Workyard gets a sense of how an engineer would actually fit into their engineering org, and conversely, candidates get a real sense of what it would be like to work as an engineer at Workyard.

How companies get this wrong:

• Projects that are really toys, completely disconnected from work products.
• Projects that are overly contrived to test for something specific — how candidates commit, for instance.

3. Leave No Room For Scope Creep

When an engineer is on your team, they are trying to ship a successful product. When an engineer is working on your take-home test, they are trying to impress you. No matter how realistic your take-home projects are, this difference will change the way engineers approach them.

One of the biggest risks is scope creep. If you give an engineers a project with no limits like “build a frontend that consumes our API and has X key features,” the scope is theoretically infinite. While on a real project, the same engineer will (hopefully) be focused on shipping features that serve your business objectives, on this project, the engineer will be worried about impressing you. Adding extra features, using unnecessary new frameworks, rewriting your boilerplate in TypeScript — it can get out of hand quickly.

Even if an engineer doesn’t add hours of work to the project to try to go above and beyond, they’ll be dealing with the stress of guessing whether you secretly want them to. Were you being purposefully vague to see if they’d overdeliver?

Prevent this by clearly stating expectations around your project. Frame.io, the popular cloud-based video post-production platform, is a strong example of this with its frontend take-home, which you can see in this GitHub repository.

The project itself is fairly straightforward. They provide a working autocomplete search component (think Google search) and ask engineers to implement two new features. They provide a working demo of the basic app in their boilerplate — which you can install in less than a minute:

Along with this demo, they also provide specific expectations around engineers’ work on the project:

- The component should be reusable. It should be possible to have multiple instances of the component on the same page.

- The “States” example that uses a data array should continue to work.

- The component should accept any HTTP endpoint, not just the https://api.github.com/users example above.

- Your component should work correctly in Chrome, don’t worry about cross-browser compatibility.

- You can use small DOM helpers like jQuery or utilities from Lodash, but not larger libraries/frameworks like React, Angular or Vue.js

- You can modify all parts of the existing code, but you don’t need to do that to provide a great solution.

- Document your component in SOLUTION.md.

Finally, they instruct engineers to stop working at the four-hour mark and explain any extra work they would do if this were a real work product. In this way, the Frame.io team leaves room for engineers to go above and beyond without actually dedicating the time to spinning up extra code.

How companies get this wrong:

• Projects with vague deliverables, leaving the door open for massive scope creep
• Projects judged by unclear criteria — if extensive commenting is a priority for you, say it
• Projects that are impossibly large. If you’d expect your full-time engineers to need a full day to ship it, it’s too big

Give Engineering Candidates A Real Code Review

The other mistake startups make with take-home projects doesn’t have anything to do with the composition of the project itself. Instead, the problem lies in the review process. For candidates, one of the most demoralizing experiences is spending hours on a take-home that reflects their best efforts, only to be called into an onsite interview where the manager clearly never looked at their code.

If you give an engineer a take-home and bring them in for an onsite, respect the time they put into the project. Talk through their code with them, question their decision-making process, and get a feel for how they think about software.

Your take-home shouldn’t be a stand-in for a technical screen. It shouldn’t be just another hoop for engineers to jump through. Your take-home should save you time in the interview process, help you better evaluate your candidates, and take some of the stress off of engineers — but it can only do all of that if you’re as invested as your candidate.

Originally published at angel.co.