Code challenge is a questionable “time saver” at the expense of candidate
I landed my first payed software developer job, when I was still in school. I think I was 16, or maybe 18. I went to “Arbeitsamt” (german employment center) to ask if I could get a job for summer vacation. It was a shot into the blue, I did not expect anything out of it and the clerk at “Arbeitsamt” gave me to understand that I was spot on. He had nothing in particular for me, but than we started chitchat and he sad:
If you think you can handle it, here is a medical doctor, who needs help with MS Access database…
I was young, bold and into Visual Basic, so how hard could it be? I took the offer and ended up working on this database for the summer vacation.
Since than, I had many different jobs at many different places:
- I had plenty student associate jobs, while studying computer science
- I was a Software Developer / Architect at an online bank (at-least this is the story my fancy business card told)
- I was IDE developer at a multimedia agency
- I worked at two consulting companies and was placed in big projects at big companies
- I built mid core mobile games and SDKs for a bigger independent games company
- And since 2016 I worked on multiple iOS apps as freelance developer
You can imagine, that I had a few experiences with technical interviews over this period of time. And there were also different interview styles, but they were mostly concentrating on adhoc questions and your previous work. But lately I see (in my opinion) disturbing trend. This trend is: code challenges!
From my experience I can identify two types of code challenges:
- Here is an online IDE, couple of minutes time and multiple trick questions we found in a smart book.
- Here is an artificial problem, solve it at home and solve it in a way that we think is a great way of solving it.
Here is my personal opinion on this kind of interview:
It sucks!!!
First of all, it sucks because you are offloading time you need to spend evaluating the candidate on the candidate.
In approach number 1. you just reduced your involvement almost to 0. The candidate needs to spend time solving the trick questions, where you will just get a report of how they scored. This is amazing except, you need those scores to be significant, so let’s make the problems extra tricky, so candidates need to spend weeks of preparation to be able to pass those extra tricky questions. This way we see that only a fraction of candidates were able to solve those tests, meaning that those people are the best people for the job.
The problem is, those people were the best people to solve those tests. It tells you nothing about, how they will fit in your company and help you evolve your code base / product.
We hire people mainly for two reasons:
- We need someone who will be a good cultural fit and is smart.
- We have a particular problem we need help with.
First reason is best way for sustainable company. It concentrates on long term goals, building a company with great culture populated by smart people.
Second reason is more of a short term gain. We have a problem, it needs to be fixed. Do you know this technology? Yes? You are hired! This cultural nonsense we will figure out later… You also don’t have to be smart in general, you just have to have enough experience with the problem we need solving now.
Coming back to the coding challenges. What do you think, is a person who was able to crack your coding challenge is — smart, or experienced?
My money is on experienced. A test is a test. If you keep doing similar tests over and over again, you will be able to pass them. So experience is key.
It tells you nothing about their ability to think out of the box. Communicate with others. Picking the simplest solution. Dam! It does not even tell you how well they perform under pressure. If some one have enough experience solving trick problem under time constraint, this task becomes a routine for them.
Now, is the second type of code challenge (home work) better?
In my opinion, it is better, but it has it’s own issues.
Again, the main objection of the “home work” is to make it as comfortable as possible for the interviewer not the interviewee. You want them on your turf. You come up with some kind of an artificial problem. And you have your own opinions on how this problem should be solved. So a “good” candidate needs to strike the nerv. Solve it in a way that you approve.
This approach has some degree of “prove of cultural fitness” which is at-least something in comparison to the first approach.
However I believe the main problem is still the motivation behind the code challenge. The main motivation is to save time. I saw so many blunt and dumb code challenges, where you are bored out of your skull just by reading the assignment. Specifically if you have open source projects, which show you solve more complex problems. And than there is this second layer, which is much more important — the expectations of the interviewer. And those are mostly not expressed in the assignment. This results in a question:
How should I over-engineer this thing in order to be accepted?
This is a degree of paranoia, which should not be part of an interview process in my humble opinion.
All in all I hate code challenges, because it is rare that a company is able to produce an interesting and engaging home work. And if I see those trick question you need to solve in X minutes, I directly turn it down.
If you are hiring people, please think about the experience on both sides. I understand that you don’t want to waste time on interviewing “the wrong people”, but you should keep in mind that automation is rather a bad idea, when you trying to solve interhuman communication issues. Also keep in mind that people on the other side, don’t want to waste their time either.
PS: Do you know, what is more annoying than spending hours, or days on artificial problem and being rejected? Spending hours or days on artificial problem and being rejects without proper feedback!
If you ask people to invest time on your code challenge, please have at-least the decency to give feedback and explain, what exactly was wrong. This way it becomes a teachable moment and not a complete waste of time for the candidate.
