Running Great Tech Interviews: The Art of the Follow-up Question

We can interview clearly and decisively. The key is to ask good questions with good follow-ups. We need to know what we’re looking for and how we’ll find it. I'll share some great questions and show how to follow up on them effectively.

The Aims of the Interview Conversation

An interview is not just any conversation. It should be a purposeful conversation. Our main role as interviewer is to direct what the candidate talks about.

Our direction can give the candidate opportunities to show whether they have what we're looking for. That means knowing what we’re looking for.

Typically the questions we should be looking to get answered for ourselves are:

1. Can the candidate produce solid relevant technical work?
2. Can they communicate about that work clearly?
3. Would they work in a focused and sensible way?
4. Are they comfortable making compromises and trade-offs?
5. Do we feel we could work with them?

So how do we best lead the conversation to get answers?

Leading the Interview Conversation

Start from the CV or any of the candidate’s work. Look for suggestions that this candidate has done something relevant or interesting. Get the candidate to tell stories and explain their decision-making.

Keep moving the focus to work that the candidate has done and how they’ve approached problems. We want to avoid focusing too much on companies they’ve worked for or the context of previous jobs.

There’s a style of questioning we can use to direct the conversation.

Here's some great starter software engineering interview questions:

• Is there a piece of work from that time that you’re particularly proud of?
• Could you give a brief example of something that you personally contributed to the team’s work?
• Did you learn a lot in that role? Any lessons in particular that you learnt?
• Could you briefly summarise your experience with technology X?
• What was the toughest challenge you faced working on/at Y?
• I don’t know much about Z. Could you explain the essence of it for me?

These types of questions direct the candidate on what to talk about. And they give hints on how much detail we want.

We shouldn’t feel bossy asking questions like this. Our tone should be friendly and approachable. Do let the candidate ask what you mean. 

Candidates typically like the interviewer giving them clear direction. 

If we’re not clear what the candidate is talking about, we should tell them that. Make sure the impetus is on the candidate to make themselves understood. If you can’t understand them then you can’t hire them.

The point is not to find out if the candidate is a good communicator. It is to find out if they can communicate with you. Ideally about problem-solving close to your work.

Drilling Into Details

Let’s see what this approach can look like in detail.

Candidate: I introduced a new library for mapping code entities to the database. It sped up some queries by 40%.

You: Were there any challenges with introducing that?

Candidate: When I first looked it wasn’t compatible with another library we were using. But we were planning to upgrade that library soon anyway so brought that upgrade forward and it worked with the newer version.

The initial story is a good start but doesn’t show you problem solving. The follow-up leads into the challenges. The handling of those challenges reveals sensitivity and suggests good collaboration skills.

If this were a real conversation you’d name the library and what the incompatibility was. This requires some common ground. You want to direct the discussion so that you can ask good follow-up questions.

If the candidate doesn’t remember the level of detail you want then you can ask for a more recent example.

Let’s consider another example.

Candidate: I led the implementation of a new feature to generate business insight reports.

You: How did you ensure the reports offered the insights the business was looking for?

Candidate: It took a couple of iterations to get them right. We agreed initial specs but they were a bit rough and the business wasn't very clear from the beginning. In the second iteration they got more excited and could guide us more clearly.

Again, the initial story doesn’t demonstrate much. It asserts an achievement but it doesn’t demonstrate. The answer to the follow-up leads us into an appreciation of stakeholder management and requirements solicitation.

Getting into detail does not have to be based on stories. You could ask the candidate to draw out the design of the last system they worked on. Or work through a small coding problem like FizzBuzz.

Being Organised for Good Software Engineering Interviews

Often we’re under time pressure. We have a mountain of candidates to get through. We’ve got loads of other things to do.

If we’ve not looked at the candidate’s CV properly then best to be upfront about that. Ask the candidate to walk you through it.

When we’re rushed then we tend to go with the flow. That leads to letting the candidate talk too freely and without structure. We need to be directing the conversation.

We want to think before the interview about what we’re going to probe on. Maybe we want to know about experience with technology X. Maybe we’re not sure they’ve worked to tough timelines. What could the candidate say that would answer or confirm those concerns? 

Think about what kind of stories you’re looking for. Imagine yourself interviewing for the role. Consider what work you’re proud of. Or some great work that a colleague did. You’re probably looking for stories a bit like those.

If you’ve used a technical test then it’s valuable to review it well before doing a run-through with the candidate. Having a few notes going into the interview makes a big difference.

When a candidate scores well on a technical test it certainly shows some skills. But you gain more confidence from walking through the thinking. Then you see the skills in action.

Having a Strategy for Interviewing Software Devs

It might sound like I’m recommending behaviour-based interviewing. Talking about specific scenarios is certainly the essence of behavioural interviewing. But I’m only recommending a similar style of questioning. It’s a tool to get into enough detail to reveal problem-solving skills.

You don’t want to let the interview become a chat that doesn’t give you answers. You want to have a strategy and then improvise a bit within that strategy.

Part of having a strategy is knowing what you’re looking for. Think about what you’d need in order to say yes to a candidate. This forces you to think about what you’re really looking for.

Confusion often enters the process around what is needed for the role. Jobs ads may ask for all the relevant skills and background for the job from day one. This is rarely necessary. Aptitude for learning and problem-solving should be given more weight.

It’s ok to list all the relevant skills on the job ad. But don’t let that dominate the interview. There’s a risk of turning the interview into box-ticking of technologies and experience. That won’t really give you confidence.

To get confident about your decision you don’t need to tick boxes. Or even be told a good story. You need to see that this person can think about relevant problems and share their insights with you.

A candidate with impressive stuff on their CV might get you excited. But confidence comes from having a real discussion about challenges.

The Essence of Good Interviewing

Good interviewing is about getting to the point. Unfortunately, the point is complex and you can’t just jump there.

We want the candidate to talk us through some relevant work. We want to see from their explanation what exactly they did, why and what the challenges were. There are almost always compromises and trade-offs. 

Hopefully, the candidate will have many stories like this. We need to prompt them because they don’t know which ones to tell. And we need to nudge them to explain everything clearly for us.

The onus is on the candidate to show relevant problem-solving skills. The onus on the interviewer is to get the right level of detail.

Getting into challenges and compromises is important because it shows that problem-solving really was involved. We know the work that needs doing involves real obstacles. We don’t fully understand successes without understanding obstacles.

We want to be clear what we’re looking for. And clear about what will count as demonstrating what we’re looking for. This will give us confidence that we’re making the best decisions that we can.

Title image by author using elements from Peggy_Marco and OpenClipart-Vectors on pixabay.