Recruiting Software Developers – Coding Tests

For the past few months, I have been interviewing with several companies. In all cases, one or more coding tests were included. I have also been on the interviewing side, evaluating a coding test. Here are my thoughts on the process.

Are coding tests needed?

When I started out as a software developer, nobody did coding tests as far as I know. You just talked to the hiring manager for an hour, and if they liked you, you got an offer. These days though, many companies (but by no means all) require you to do a coding test. Why is this?

I think one reason is that they want to make sure the candidate can actually code. Even though I have never seen examples myself, I have read many stories of candidates that can’t even code on a FizzBuzz level, so I have to believe that coding tests are necessary.

Another reason is that it is a way to make sure the candidate is comfortable with common data types and algorithms. For example, if you can write code to traverse a tree, you are able to deal with recursion and pointers.

Both of these are valid reasons in my opinion.

Problems With Coding Tests

Examples of coding test problems are: reverse a linked list, find the median of two sorted arrays, generate all valid combinations of n pairs of parentheses. These are all well-defined problems. More advanced problems often require some clever algorithm, optimization or data structure.

In contrast, typical tasks at work rarely require clever algorithms. However, you almost always need to understand existing code first, so you don’t break anything when adding the new feature. Clear and maintainable code is important, as are good tests. You can discuss with colleagues to figure out the best solution. If there is time pressure, it is usually measured in days or weeks, not in minutes or hours. Sometimes tasks are not so well defined either. You can decide where to add the functionality, and you can consider whether it can be solved by modifying an already existing feature instead of adding a new feature.

So the coding tests are almost the opposite of what you do day to day as a developer. Because you aren’t naturally exposed to these kinds of problems when working, a whole industry has sprung up around coding tests and interviewing practice, with sites like Leetcode and HackerRank. But if you need to practice for an interview for a job, even if you have worked in a similar job for your whole career, something seems broken.

Despite this, I actually like coding problems. I like the clever algorithms, and how you can solve difficult problems efficiently in a few lines of code. I sometimes do them for fun – for example, I did many of the Advent of Code problems last year. But they are not representative of what I do at work.

However, not all coding tests have these problems. Some are more like actual work than others.

Different Kinds

I have come across these four types of coding tests. In all cases, I was able to choose the programming language I am most comfortable with (Python, since that’s what I use most at work). This reduces the stress a lot. Even though I have programmed professionally in several other languages, I am not as fluent in them anymore as I am in Python.

Whiteboard. I have done one whiteboard coding test (before the pandemic), and for me this is the worst kind. First off, it is very far from coding in an IDE on a computer. Secondly, my typical way of solving problems is very iterative. I code a small part of the solution, then test it, and correct it when necessary. Then I add another small part of the solution. On the whiteboard you can’t work iteratively like that. Yes, you always have to have some idea on how to solve the whole problem, but getting all the details right usually takes several iterations for me.

Timed, auto-graded. One test I did consisted of a series of programming tasks at an interviewing site. There were maybe seven tasks of increasing difficulty. All the tasks had time limits (from 5 minutes to 35 minutes), and test cases that should pass. It was quite stressful seeing the timer counting down – I felt like my brain only operated at 50% capacity. There was an online editor which was quite alright. However, since I am used to coding in PyCharm, I sometimes mistakenly tried to use shortcuts or commands that were not available. However, I understand why companies like tests like these. All candidates get the same tasks, and there is a report on how they did. But I wonder if the code quality is assessed beyond whether the test cases passed.

Take-home assignment. These have a tendency to require a lot of work. Partly because companies underestimate the work involved, and partly because they are usually more open-ended than Leetcode-style problems. The good part about them is that you can use your regular development environment. Also, you usually have at least a day to do it, so no down-to-the-minute deadline. As an interviewer, going through the code with the candidate can reveal a lot about how they code. You can discuss the structure of the code, names, tradeoffs, tests and so on. One potential problem is that you can’t be sure the candidate did all the work without any help.

Live coding. Coderpad seems to be the standard platform for this type of coding interview. You solve a problem you are given while the interviewer watches. I like this format, because you can have a conversation with the interviewer as you develop your solution. You can ask clarifying questions, and if you get stuck you can get some help. Sometimes it can almost become a pair-programming session.

Best Experience

As I mentioned before, many coding tests are far from what you do day to day. However, it doesn’t have to be that way. The two best coding tests I came across were both realistic problems. In one case the input was lines from a log file. The task was to develop a script to find which user_id occurred the most times.

The other case was a program for extracting some word properties from a document of text. There was an existing, but buggy, solution, and a number of failing test cases. The task was to understand the existing program and fix the bugs in it by using the test cases.

Both of these tasks were done as live coding sessions in Coderpad, and were close to something I could have been doing at work.

How I Prepared

Even though coding tests are not representative of actual work, they seem to be here to stay. Even though they are far from ideal, maybe they are the best thing we have. Anyways, before this round of interviews, I decided to take a few weekends and prepare. I had already read he classic book in the field, Cracking the Coding Interview. I also have a book called Elements of Programming Interviews, which has a lot of good examples with detailed solutions. I used both books to review various data structures, algorithms and problem-solving strategies.

However, for practicing solving problems, I used Leetcode. In the past, I have used HackerRank as well, but I like Leetcode better. Leetcode has collections of practice problems divided into categories like arrays, strings, sorting and searching, trees, dynamic programming and so on. The problems are rated as easy, medium or hard. I mostly did easy ones, but they were usually difficult enough I thought.

I signed up for the paid plan, which gives you access to the solutions to all the problems. This was money well spent I thought, because there were usually several solutions for each problem. Reading through them taught me more than simply getting to a solution that worked. There were typically solutions in one or more of Java, Python and C++.

Another feature in Leetcode that I really liked is that each problem is tagged with companies that have used that problem in interviews recently. So if you are interviewing at Google for example, you can sort on the most common problems from Google interviews in the past 6 months (as reported by users).

There is also a good blog post I found that talks about various Leetcode problems: Passing Your Senior Engineering Coding Interview

Conclusion

Whether you like it or not, many interviews these days will include a coding test. The tests can be more or less similar to the work you do as a developer. Preparing for them by reading and practicing on Leetcode helped me feeling less stressed about them. What is your experience with coding tests? Let me know in the comments.