Engineering whiteboard interviews: yay or nay?

Originally published by Lynne Tye on September 25th 2018

@lynnetyeLynne Tye

Software engineers hate whiteboard interviews. How do I know? We continually tell complete strangers just how much we hate them. But it’s not that simple.

The problem is that we often conflate several issues: using random CS trivia/riddles to assess candidates, interviewers deliberately creating a stressful environment for interviewees, competent humans who happen to interview poorly, and the job search sucking in general.

“Whiteboarding” has become too large of an umbrella term, one that groups together everything that’s wrong with the interview process. The truth is more complex, which is why I talked to hiring managers at several tech companies to get their opinions.

Are whiteboard interviews the best way to evaluate engineering candidates?

Jamie Karraker

CTO at Alto, previously engineering at Facebook (Parse).

I don’t believe in the traditional whiteboarding questions on data structures and algorithms, at least not for our business. An interview should be designed to test as closely as possible for the skills an engineer would be using day-to-day in the job. For us that means being extremely productive, given we are an early stage startup and need to build a lot of product and make a large impact with very few engineers. But for Google they have to be able to solve very few, but very complex, technical problems, and productivity is less of an issue given their limitless resources. So for them the whiteboarding question might be the right fit, as it is an efficient and scalable way to test for performance on solving complex technical problems.

Eugenia Dellapenna

Engineering manager at Medium, previously engineering at Pivotal Labs.

I don’t think they’re the best way to evaluate candidates, but I also don’t think they’re as useless as some blog posts make them out to be. I actually think that they’re the best way to ask certain kinds of interview questions, such as high level system design questions, but that they aren’t the best way of asking coding questions since it’s pretty different from how people are used to writing code on a day-to-day basis.

Nowadays, I tend to favor using collaborative coding environments, like CoderPad, for coding interviews. CoderPad is nice because it allows you to run the code, so you can quickly test the code with different inputs to see if all edge cases are handled. This also allows you as the interviewer to see how the candidate debugs real code and how well they’re able to interpret things like compiler errors and runtime exceptions. It usually takes a bit longer to complete interview questions in this coding environment though (because of the time spent debugging) so you tend to not have as much time to build on the initial question or discuss further improvements, which is one caveat.

I still usually give candidates the option to write their solution on the whiteboard if they really want to, since some people specifically practice that when they’re preparing for interviews, and I want people to feel as comfortable as possible. I think in general, it’s your job as an interviewer to set candidates up for success so you can get as much signal during the interview as possible. Doing that well is more important than if you’re asking the question on a laptop or a whiteboard.

Brian Wheeler

Senior Director of Engineering at Braze, previously engineering at Microsoft.

I’d say “best” is an overstatement, but I do think they can be a valuable component of an interview process. When done properly they can be a reasonably consistent evaluation of problem solving ability, communication skills, and preparedness. Some questions are not suited for whiteboard interviews, but for all others the more important factors are what you ask, how you ask it, and how you dialogue.

Dan Erickson

VP of Engineering at Eaze, previously CTO at Getable and engineer at Yammer.

No, not at all. Whiteboard interviews test one thing well: How well does a candidate code on a whiteboard. Engineers on my team never have to code on a whiteboard (whiteboards are really bad at running code), why would I make candidates do something that I don’t ask the engineers already on my team to do?

Some argue that whiteboard interviews can help you understand a person’s “soft” skills (a term I hate because it downplay’s the importance and difficulty of emotional intelligence skills). In my experience, there are better ways to evaluate these skills, and often times directly asking the candidate to give examples of these skills is more effective.

Desmond Brand

Director of Engineering at Flexport, previously engineering at Khan Academy and Microsoft (Bing).

Whiteboard interviews might not be popular with the median candidate, but I think Kevin Lacker nails it in the example at the bottom of this blog post.

There’s nothing special about the medium of a whiteboard itself, however. At Flexport, we give people the option to write code on their own laptop if they prefer, and some take us up on that. This works better for those who use a TDD-style process or who just like to insert text before other text :)

But how and where code is written is incidental. The important part of these interviews is communicating (using words & diagrams) about a problem and approaches to solving it, and then also communicating (using words & code) about teaching a computer to solve that problem too.

There are more drastic changes we’ve experimented with, e.g. a take-home project, but those tend to impose an asymmetric time investment upon the candidate, which some people greatly dislike. So in practice, projects can only be an option at best. However, having different styles of interview can be problematic too because it increases noise in evaluations of candidates, which then increases the risk that unconscious bias creeps into hiring decisions.

Sandy Jen

Co-founder of Honor, previously founded Meebo and sold it to Google in 2012.

From a functional perspective, it’s not very useful… no one will be producing code on a whiteboard as their primary job. However, it is a good way to learn how a candidate communicates, expresses ideas, and takes feedback. The expectation is not to ace the whiteboard question, but to learn more about you as a potential team member.

David Chang

Engineering Manager at NerdWallet, previously engineering at aboutLife.

There’s no one best way to evaluate engineering candidates. All interview processes have their own strengths and weaknesses, and it’s important to understand how much (or how little) any given question can tell you about a candidate.

At NerdWallet, we usually conduct architecture and design questions on a whiteboard. It helps us to better understand how candidates structure and communicate complex technological solutions and the broad strokes of the design are more important than the specifics of any particular line of code. We also normally ask a couple laptop coding questions, for obvious reasons. Recently, we incorporated a pull request question, because working with other people’s code and giving feedback is a key part of being a productive engineer at NerdWallet and we were missing that from our previous interview process.

None of the questions are perfect, though. Some engineers feel uncomfortable designing completely new systems on demand, coding questions may just happen to hit on a candidate’s blind spot, and understanding a PR for code you’ve never seen before can be challenging. We’re big believers in feedback and data-driven decision making, so we’re always working to better understand what our interview process can tell us about a candidate and determine ways in which we can improve it.

Zeesha Currimbhoy

Director of Product Engineering at Branch, previously VP of the Augmented Intelligence Team at Evernote.

I have spent my entire professional management career debating what the best way to interview candidates is. In my ideal world, a candidate should never have to “prepare” for an interview and an interview is just as simulation of a day in the life at work. In this vein, I have played around with different formats, the take home challenge, the white boarding interview, the algorithms and data structures sections, a live coding session, a debugging session and the list goes on. I have a lot of opinions on this topic (much like any other manager). But to sum it up, whiteboard interviews, in my opinion is a great “tool” to use while evaluating candidates.

Thats right, in my opinion a white boarding session should be thought of as “a tool” to enable you to problem solve with a candidate. What’s great about a white boarding session is that it is a blank piece of canvas, where the interviewee can effectively think out loud, collaborate, draw diagrams and also brainstorm with the interviewer. It doesn’t have friction between your thoughts and being able to communicate them. At the same time, it shouldn’t be the only way to interview candidates and is definitely not a one-size-fits-all. An effective interview is an assessment across several different dimensions with different ways of evaluation. Whether we like it or not, a whiteboard represents a large part of technical communication not just in an interview but in day to day operations as an engineer.

Larry Salibra

Engineering Partner at Blockstack, previously founded Pay4Bugs.

I don’t think whiteboard interviews are particularly useful in that they don’t reflect the reality of performing the job. Open-ended coding challenges that involve using our software to build something give us much more information about the skill level of a candidate and his or her interest in the project.

As an open source project, we’re lucky in that candidates really interested in our project can prove themselves by sending pull requests or building apps on our platform — in some cases this means we’re comfortable making an offer without asking the person to do an additional coding challenge.

Derrek Harrison

Engineering Manager at Samsara, previously the Director of Engineering at Kinetic Growth.

Great engineers have many tools to choose from when solving a problem — great companies also have many tools at their disposal when searching for and evaluating talented team members. A whiteboard interview is an excellent tool when the candidate needs a large canvas to diagram out a problem or design a system. This is particularly true when asking a system design/architecture question or asking the candidate to describe the interaction between multiple systems.

However, whiteboard interviews are not a fitting format when testing a candidate’s ability to write code. If you take away the ability to debug and test during an interview, you are left with a scenario that is not an accurate representation of day-to-day engineering. A collaborative coding environment is a better choice here — it gives the interviewer the ability to clearly define a problem in writing and gives the candidate the ability to write, debug, and test in a familiar setting.

We strive for an interview format that allows a candidate to shine and gives us the best possible signal on how the candidate would perform as a member of the team. This means that sometimes, but not always, a whiteboard interview is the right format.

Adam McKerlie

Director of Engineering at G Adventures, previously an Engineering Manager and Developer at G Adventures.

After years of iterating on our interview process, testing out different ways to evaluate a person’s skill and personality, I’ve found that whiteboards add very little to the overall interview process. They can be useful at understanding how someone works, but more often than not, I’ve found them to weed out really good candidates who haven’t had the opportunity to do whiteboarding interviews before. I much prefer to talk with a candidate over a problem and have a conversation with them, similar to how we’d discuss a problem in real life when scoping out a project. I find people are generally more at ease with this approach and don’t get tied up on syntax, their messy writing, or talking aloud while “coding.”

Looking for a new job?

Everyone quoted in this article is a hiring manager at a company on Key Values. You can learn more about their engineering culture by reading their team’s profile, and if you’re still interested, I encourage you to reach out to them directly.

Originally published at www.keyvalues.com.

Engineering whiteboard interviews: yay or nay?

@lynnetyeLynne Tye

Looking for a new job?

Related

@edward-moonEdward Moon

@ani-alexanderAni Alexander

Tags