Photo by Clem Onojeghuo on Unsplash

As a contracting developer, I go for about four or five interviews a year, and I’m on the hiring side of the desk about the same number of times.

I’ve worked side-by-side with some truly great developers, and a few who were not so great.

Now, it’s not easy to predict how good a developer really is in the few short hours you have during the interview process, but I’ve got an opinion or two about what works well and what doesn’t.

And you’re about to read them.

Warning: I disagree with a lot of conventional wisdom so I’m afraid this is one of those “you’re doing it wrong” blog posts.

How to assess a developer

I want you to take a moment and think of the top three developers you’ve ever worked with.

Thank you.

Now what made these superstars so super?

You’ll probably think in terms of productivity and quality. Those who were keen to do things The Right Way, and were enthusiastic about discussing ‘The Right Way’ with others.

You might also recall with fondness the ones who took the time to help others, who shared their knowledge. Maybe even being nice counts for something.

Now think of bottom three developers — those that you would definitely not hire again.

Why did they suck so badly?

Maybe they had that bare-minimum mentality and didn’t take well to feedback. Maybe they were slow to produce results and wouldn’t ask for help. Maybe they just weren’t interested in being great.

So, how do the characteristics from above align with how you currently assess developers during the interview process?

In my experience, there’s quite a disparity. We want people that are productive, passionate and enthusiastic. But we send out coding challenges, play 20 questions and draw a B-tree on a whiteboard.

Let’s look at some of these techniques in detail…

Coding challenges (probably not great)

I’m 80% confident in my opinion that a ‘coding challenge’ is a waste of time for both parties.

Story time: a while back, I worked for a company where the CTO decided we should start administering a coding challenge as the first step in our interview process.

I was tasked with devising a test and asking candidates to complete it. A week later, I returned to the CTO with a (digital) stack of completed tests that we went through and assessed together. Of the seven candidates, we (he) decided that 3 were worth an interview.

I then revealed to him that actually these weren’t tests from candidates. I had instead given the test to our existing seven developers.

“So,” I said, with the smuggest face you’ve ever seen, “if you really believe that a coding challenge is a reasonable basis for elimination, then you are logically obliged to fire the four developers who you considered not even good enough to get an interview. Oh and one of them is your nephew.”

I was immediately fired for insolence, but I had made my point — it’s so easy to rule out a great developer before you even speak to them.

Think about that, before you even speak to them. It’s nuts!

I once heard a hiring manager proudly proclaim that only 12% of candidates make it through their coding-challenge stage. That’s like a toddler announcing with great glee that they did a doo doo in their pants. I’m pleased that you’re so pleased with yourself, but I’m not sure it’s something to be proud of.

And I think the problems with coding challenges go further than being an ineffective predictor of a great developer.

Imagine this: a candidate has applied for three roles and you’re the only one that set a two hour coding challenge. The candidate has had initial phone interviews with the other two and things are looking promising. They don’t like spending hours writing code to be looked at for 7 minutes then thrown out, so they put you off for a bit to wait and see how the other roles pan out.

You never hear from them again.

So not only might you rule out an excellent developer by poorly judging the results of a challenge, but you could miss out entirely because the person simply didn’t want to do your challenge.

Now, you might think “if they really wanted to work for us they wouldn’t have been scared away by a coding challenge”.

How awful!

Imagine asking candidates to pay $100 to get an interview. Would you say “if they really wanted to work for us they would’ve paid the hundred bucks”?

Maybe they don’t have $100 to spare. Maybe a single parent with a couple of rug rats doesn’t have two hours to spare?

Regardless of your feelings about level playing fields, it might be your loss, and you’ll never see what you’re missing. That should be reason enough to be wary of coding challenges.

To me it’s quite clear, this practice is simultaneously unkind to candidates and bad for the prospective employer.

On a personal note, when hiring I don’t give coding challenges any more, and when looking for work (the Sydney market is very good for developers), if there’s more than three or four promising opportunities from employers that actually want to talk to me, I’ll probably ignore anything with a coding challenge brick wall as the first step. (Secretly I do the coding challenge anyway because I love that shit, I just don’t send it in out of spite.)

OK so now you know what I think about coding challenges, but for the people who aren’t swayed to my words of wisdom, may I offer some guidelines for your coding challenge that — to be clear — I think is a dumb idea:

Keep it short. 30–60 mins. Enforce this (e.g. ask the candidate to send an email when they’re ready to go, send the test instructions and request that they reply with the code in the allotted time).
Keep it focused and close to the kind of work you want them to do (if you’re hiring for a reasonably focused role). Stick to things that candidates should have done before.
Put more activities in the test than can be done in the allotted time.
If possible, appraise the results anonymously. One benefit of this is to sidestep potential bias, so you might as well take the opportunity to have your devs review the code anonymously.
Have your existing developers take the test to get a baseline (yes, I know no one will actually do this).
Decide up front how much you’re going to read into the results. Will only 10% get through, or are only ruling out the truly terrible ones?
Don’t ask people to take the test if you don’t think they’re going to get the job. I have heard and despise the following words: “I don’t think they’re a match … well, send ’em a test anyway and we’ll see how they do”. In other words, don’t be a jerk.

Quiz questions (a bad idea)

After the coding challenge often comes the quiz questions.

We google “interview questions for [programming language]” and boom, behold: the perfect set of questions that will produce a nice neat bell curve, from which we’ll pick ’em off the right hand side.

But here’s the problem. A frontend developer, for example, doesn’t need to know everything on some “list of interview questions for JavaScript developers” in order to be a good frontend developer.

There’s about 10,000 things you could know (I made a list!), but we mostly just learn what we’ve had to know in our past roles.

Maybe a candidate has never built a website that could only be accomplished by using ‘prototypical inheritance’ in ‘strict mode’ with ‘curried functions’. (I’d suggest that’s a good thing.)

You might be tempted to think that on the whole, a candidate’s answers to 60 questions will paint a picture of their skill level.

I think that’s wrong.

But here’s the good news: you don’t need to take my word for it, this is a falsifiable hypothesis.

As with the coding challenge, you can take the test to your existing developers (or even just shout out a few choice questions — “hey who here can tell me the difference between an attribute and a property?”).

Then you have the two bits of knowledge required to see a correlation. If the result is something like this chart, then I’m well and truly wrong:

The correlation that must exist for quiz questions to be a reasonable approach

But what I think you’ll find is that you have some great developers who don’t know the answers to many of those questions, and that you have some very knowledgeable developers who seem to spend a lot of time in the kitchen watching basketball on their phone (yes Bill, we all notice — perhaps get some headphones?).

I think the reality looks more like this:

We’ve all met one of those dots at the top of column 2.

And to be clear, when I use the word “good” on the x axis, I’m talking about the qualities of my favourite developers: productive, writing high quality code, interested in sharing, discussing, growing.

The lack of correlation is particularly easy to imagine with the example of a developer moving from one language to another. Picture someone with a decade of Python experience that is just getting started in the front end world. Their memorised knowledge of specific APIs might be spotty, but that’s of little concern if they’ve got the qualities of a superstar developer.

I once had a phone interview with Facebook. I prepared by Googling “facebook interview questions” and was amazed to discover that the questions I found were exactly the questions I then got asked in the interview. “What’s the difference between

call

and

apply

”, “explain event delegation”, and other such nonsense that millions of excellent developers wouldn’t know, but could learn in an afternoon.

I like to casually sidle up to other developers and whisper-ask them what the difference is between a statement and an expression, or a method and a function. I think some people would be astounded to know that there are experienced developers who shrug their shoulders at such questions.

I’m quite sure that if you try this yourself, the more people you ask, the more you’ll begin to doubt the relevance of quiz questions in the interview process.

(It’s funny how hard it is to imagine that people can get around not knowing the things that you’ve known for a long time. I mean, can there really be people that don’t know what a German Shepherd is? Is there really someone who is right now googling “german shepherd” and exclaiming ooooooh, it’s a dog!)

The obvious retort to the above is that “Facebook is doing just fine” and that’s worth thinking about some more. Because you can miss out on hiring the best developers, wind up with the theoretical runners up, and never know what you’re missing. Things will appear to be going just fine. It’s like code that fails silently out in the wild, you may never know.

Testing for passion (a daft idea)

Beware the question that rewards dishonesty.

Some companies will ask candidates to write a cover letter (or even an essay!) to apply for a role. They’ll ask the candidate to describe why they’re super pumped about the company’s core values.

What idiocy — are you hiring a novelist or a developer?

A less ridiculous but equally misguided version of this is to ask questions like “why would you like to work for us?”

The real answer is, always, “because you will pay me to”. The answer you’ll get is whatever the developer thinks you want to hear.

To be fair, it’s an alluring idea: oh, to have a company full of people that are ‘passionate’ about our product, our purpose, our values.

Imagine the synergy!

But you’ll only get a company full of people that did a good job of telling you they are passionate about your product, your purpose, your values.

And meanwhile, you’ll have missed out on a bunch of less silver-tongued developers along the way. They’ll be working for you competitor, who didn’t insist upon the guzzling of Kool-Aid to get in the door.

And besides, you don’t want people who are passionate about your innovative insurance product or your innovative document management system or your innovative social platform. You want people that are passionate about programming, about doing A Good Job.

One of the best jobs — and best collection of colleagues — I’ve had was working for a “women’s magazine” that was complete and utter trash. The engineering team was full of passionate people, we just weren’t passionate about Karadashians, royal weddings and dingoes stealing babies.

Assessing technical experience (a great idea)

Something far more valuable than knowledge-based questions are wisdom-based questions — those that highlight a candidate’s experience.

It’s difficult to get right (I assume a big reason people just give up and go for a list of 20 questions) but well worth it.

My preferred approach is a single question: “Your job is to create a web-based email client. How would you go about it?”

The thing I like about this question is that you can lead the answer based on the position you’re hiring for. From the database, backend, API design, frontend, accessibility, UI, UX, and so on.

You can keep the candidate’s responses on track with scripted prompts: Would you start X from scratch or use a framework? Why so? This API you speak of, separate git repo or shared with the backend code? How come? How would you handle database versioning? Would you use a CSS framework? What are the benefits? What are your thoughts on accessibility? Oh you think it’s a waste of time? Interesting. And what about performance? How would you handle icons? What’s the magic number for code coverage?

The number one thing I’m looking for is that wonderful personality trait of giving a shit. If that spark is there, so much else will flow from it. Flow from the spark.

I actually find the topics tangential to the actual programming language — testing, code reviews, performance — are the most enlightening.

If you’re talking to someone and they’re getting excited about integration tests, it’s unlikely that you’re talking to someone who doesn’t care about doing a good job.

All this gives you a much richer view of the person you’re dealing with than any list of questions from the internet ever could.

If you’re not sold on the idea, I’d suggest you try it once, then at the end of a 40 minute in depth conversation about all these things, think how silly it would feel to ask “oh and one more thing: explain closures.”

Instant-fail technical questions (a satisfactory idea)

I’ve been fairly critical of technical questions, but let me be more specific: I think technical questions where you might still hire the person even if they get it wrong are pointless.

However, firing off a few must-know technical questions to rule out the bottom 20% is not unreasonable.

These are questions where you will simply not hire a person who doesn’t know the answer. This should force you to steer clear of Trivial Pursuit questions and instead look for proof of experience.

It’s hard to think of an example at the programming language level, because there’s just so much that even a good developer might not know.

As someone who typically hires developers to work on React, I stick to questions about the framework, like “explain the difference between state and props”. Something that’s on page one of the manual, but takes a little bit of experience for the difference to become clear. A question where, if the candidate couldn’t articulate the difference, I would be comfortable ending the interview and moving on to the next one.

Your approach here also depends on what level you’re hiring for. If you’re paying top dollar to get a senior developer, you can be ruthless with your questions. But if you’re hiring a junior-to-mid-level person to be with the company for years to come, it’s perhaps less useful.

Personally, I tend not to bother with these questions at all any more, instead getting straight into the “tell me how you’d build gmail” conversation and working this into my set of prompts as something specific (“As the user is drafting an email, where would you store that data?”).

Considering interview skill (a dangerous trap)

It can be difficult to differentiate between how well a candidate answered your questions and how well they ‘interviewed’ — particularly in the interview that comes after the technical interview.

If I may use myself as an example: I’ve made an effort to be good at job interviews. I’ve read and re-read Max Eggert’s The Perfect Interview. I’ve had lots of practice (on account of my insolence). I can rattle off the ‘best-of’ highlights from my career in a practised, organised manner. If asked to describe myself in three words, I will look up and to the left, pause for 2.5 seconds, then lock eyes and reply: “efficient”.

But there are better developers than I who will not be so coherent in an interview; who are nervous; who will flounder without a structure around which to talk about themselves; who will feel uncomfortable ‘bragging’ about achievements they are proud of.

You want to be confident that your process will reliably pick this second developer over the slick interviewee every time.

If a developer interviews a candidate and says “yes”, then management reviews the candidate and says “no”, it might be a sign that someone has fallen into this trap.

Or perhaps it was a result of …

Screening for cultural fit (oh my)

Making sure that a candidate is a cultural fit for your organisation is very important. You don’t want to hire someone who is rude. Or always grumpy. Or too opinionated, sad all the time, gruff. The anxiety sufferers are best avoided too (no fun at the Christmas party).

It’s interesting: the entire interview process is about discrimination (in the ‘differentiate’ sense of the word) but if you’re not careful, you can slide into discrimination (in the ‘prejudice’ sense of the word).

And when you’re screening for ‘cultural fit’, you’re attempting to discriminate, in a reasonable way (?) based on personality. But where does ‘personality’ end and ‘mental health disorder’ begin? It seems acceptable to not hire someone who is rude, it seems not at all acceptable to rule someone out because they suffer from anxiety.

But where’s the line in the sand? And can you spot that line while probing into someone’s life history as they solve fizz buzz on a whiteboard?

Story time: many years ago, I worked with a developer with Asperger’s who I will call Tom. I would describe Tom as an excellent developer. 10/10 would hire again. However I would not blame you if — upon meeting him for the first time — you described him as a jerk.

One day I asked my boss, “how did Tom even get through the interview process?”

My boss explained that he saw through the seemingly unpleasant exterior to the magnificent brain within. I asked my boss if he too had a doctor’s certificate for his sub-par personality and was immediately fired for insolence (why does this keep happening to me?)

Side note: for our many shared personality traits, I was dubbed an “honorary Aspy” by Tom — a badge that I wear with great pride.

Aspies rule!

That’s the extent of my opinions about how to pick the developers that you want to hire.

But that’s only half the story. Once you’ve found your superstar, you have to convince them that they want to work for you.

Selling yourself

There’s a (mildly) interesting duality in the process of hiring a human; you’re simultaneously buying something and selling something.

So you need to treat all candidates like superstars, right from the start. Tell them why they will love working with you at the same time as working out if you want to hire them. When you do find your superstar, with any luck they’ll already want to work with you.

An unfortunate side effect is that the ones you don’t want to hire will think you’re a big ol’ tease for coming on so strong.

Check the ‘tude

A few times now I’ve come across the “why should we hire you” question, always paired with the mentality working for our magnificent company is a privilege.

But here’s the thing, if you treat candidates like they’d be lucky to work for you, you’ll wind up with developers who are lucky to work for you.

What do you have to offer?

Can the candidate work from home? One day a week? Five? Be specific.

Dogs in the office? Dogs in the office!

If I understand the literature correctly, there are people out there with some sort of desire to “meet new people” (¯\_(ツ)_/¯). These folk might be pleased to hear that you have a ping pong table, pool table, craps table. Maybe people like to chill out after work on a Friday afternoon with some wine and crackers (the food).

Does your organisation recognise the existence of glossophobia (the dread of public speaking)? Or are people welcomed to your company by giving a short presentation about their childhood to a crowd 100 strong? I’ve known many a developer who would be wooed by the words “all public speaking is opt-in”.

Do you offer opportunities to grow one’s knowledge? Will a successful candidate get to dabble in machine learning AI blockchains? Will they be sent to the tech conference of their choosing once a year? Do you have 20% time? 10%? 5%?

Is your engineering operation a well-oiled machine? Do you have the right people doing the right things to allow developers to be productive to the maxxxx? Most companies aren’t great at this, so if you’re doing it right, sing it from the rooftop, baby.

Now, I don’t want to speak on behalf of people I ain’t, but I have been informed that some women would rather not be the only female in a room of 47 blokes. So if that won’t be the case in your workplace, perhaps a tour of the office will set you apart and help you land that superstar.

While you’re on the tour, do you have a desk set aside for your candidate? Does it have a chair with lumbar support and two 27" monitors? Will your new hire have a space to call their own, or do you have your developers lined up at a slab of white melamine like pigs feeding from a trough? (The pigs in this metaphor are feeding on the lie that open plan offices are anything other than a cost-cutting measure.)

As a type of survey, I would like to invite my developer readers to highlight which of these get your motor runnin’:

Work from home days
Social atmosphere, activities, etc.
No public speaking
Continued learning
Cool tech
Good personal work space

I’ve left out financial-based incentives because there’s nothing exciting about getting $1,000 toward a gym membership if you’re paid $1,000 less.

Or maybe there is; psychology is odd.

The process

I couldn’t quite work out where these sections should go in the article. So here they are.

Phone first

In this world, there are those who accept that they are susceptible to unconscious bias, and those who are wrong.

Given its unconscious nature, this type of bias is a sneaky one. Can you be sure that if your candidate rocked up to your office with neck tattoos, or was gigantic, or wore Crocs, that your first impression wouldn’t be tainted?

I think because of this, it’s a good idea that the first interaction you have with your prospective employee is audio-only.

By spending thirty minutes or so on the phone with someone before meeting them in person, you have a better chance of seeing past their Christmas shorts (in April!), pierced nose (through the top bit!) and MAGA hat (in case you weren’t judging yet!) when you meet them in person.

But mostly, I think the phone is simply more convenient and lets you get through more candidates, more quickly.

The matter of time

I have heard on occasion the following sentiment: “sure, we have a long interview process, but we’re willing to wait to find the best”.

You may not be surprised to hear that I think this is stupid.

If you have a four week decision making process, you’re not getting the cream of the crop, my friends. You’re getting the dregs.

If you’re in a market where there is an under-supply of developers, you need to assume the following:

The candidate that applied for your role also applied for other roles
If you want to hire them, others will want to hire them too

I like to start with the assumption that upon first meeting a candidate, they were in fact just made an offer from a competitor. The clock is ticking and I need to work out if I want to hire this person — and make an offer — as soon as possible, or they’ll be gone.

Wrapping up

There’s plenty of ways to approach the interview process, and maybe not a one-size-fits-all approach, but if you’ve got no better ideas, I’d try this on for size:

To start, a technical interview by a developer, over the phone, walking through building an imaginary app. Structured enough to compare candidates, loose enough to get a feel for experience, passion, etc.

Then an in-person interview with a manager and a peer or two. Careful not to rule out good developers based on an unfair judgement of personality.

Be prepared to make an offer on the same day if you want the person.

That’s it, thanks for reading!

May you look up at the sky tonight and see a crescent moon and think “hey, is that the one thing that all of humanity can see without squinting?” and feel a oneness with your fellow earth dwellers.

Hackernoon Archive December 12th, 2017

How to hire the best developers

@david-gilbertsonDavid Gilbertson