Developer Testing

I recently found out about the book Developer Testing – Building Quality Into Software by Alexander Tarlinder, and I immediately wanted to read it. Even though I am a developer at heart, I have always been interested in software testing (I even worked as a tester for two years).

I think the subject of the book, developer testing, is timely. There seems to be a broad trend where more and more responsibility for testing is given to developers. It follows from the move towards micro services, dev ops and the “you built it, you run it” principle. Another driving force is the prevalence of developer testing frameworks that started with JUnit and now includes many more. These frameworks encourage and help developers write automatic tests.

Despite this trend of increasing developer testing, my feeling is that many developers still don’t test their programs well enough. For example, they may test the “happy path”, but not the different error handling cases. That is why I was excited about this new book explicitly addressing developer testing.

In the preface, the author states that this is not a book for beginners, and that you should have at least three years’ experience as a developer to get the most out of it. He also writes that the purpose of the book is to give an overview of each of the topics he covers. You should be able to read each chapter in an hour or less. I think he has done a great job of covering developer testing, and the chapters are easy to read. However, I think this book is appropriate for new developers. Almost all developers who start working today will be exposed to unit testing, mocking, continuous integration and so on from pretty much the first day on the job. So in my view they will benefit immediately from reading this book.

So, what exactly is developer testing? The author’s definition is that it is everything developers do to test their code. This includes writing unit and integration tests, implementing continuous integration and providing infrastructure for test automation. Not included, in his view, is performance testing, security testing, usability testing or testing pathological cases.

What I Liked The Most

While there are many tools and techniques for testing covered in the book, the most interesting parts for me were the ones covering how to design for testability. Software that is designed with testability in mind is much more likely to be tested. If it is well tested, there will be fewer bugs. In other words, testability is a driver of software quality. Here are the specific parts I liked:

Testability.  Testability can be divided into observability, controllability and smallness. Observability means it must be easy to get and examine the result a piece of code produces. Controllability means that we must be able to put the program under test in the state we want before performing the test. Finally, the smaller the part we want to test is (in terms of features and lines of code), the easier it is to test. This may sound obvious and easy, but it isn’t. The key benefit of this book is that it puts focus on writing testable code, and this chapter on testability (chapter four) is the best in the book.

Indirect input/output, state, temporal coupling. Chapter six shows how the testability of the code suffers when there are side effects, when indirect input (like system properties, files, queues, the system clock) is used, when complex state is kept, or when the calling sequence is important.

Dependencies. Chapter nine gives many examples of how dependencies negatively affect testability, and how to structure the code so that it is still testable. Examples include working with files and the system clock. In both cases, the best strategy is to separate the data processing from the pure I/O or time operations.

Other chapters I liked were chapter three (defining the testing vocabulary), chapter 12 (definitions of the different kinds of test doubles), chapter 14 (example of test-driven development in the classic style, as opposed to the mockist style) and chapter 18 (more complex automatic tests).

What I Liked The Least

A former colleague of mine (hello Andreas) gave me some great advice on the subject of testing (also applicable to code reviews). In addition to what you would normally check, you should ask: “What is missing?”. Asking that question here, I came up with two subjects I think could also have been included.

Exploratory testing. All the emphasis in the book is on automatic checking of results, and exploratory testing is explicitly excluded. Automatic tests are important, but they are usually testing subsets of the complete system. But in the end, all parts and features must work together in the complete environment. Therefore I think it is necessary, even for developers, to make sure the system as a whole works as expected. The best way to do that is to explore how the features you have developed behave in the complete environment. The emphasis then is more on exercising the features and looking out for unusual or unexpected events, rather than checking. Since this type of testing isn’t covered here, Explore It! is a good companion book.

Performance testing. The author also excludes performance testing from developer testing. However, I think you also have a responsibility as a developer to write code that performs adequately. To ensure this, it is useful to also set up some performance testing. In the places I have worked, the developers have always been involved in the performance testing efforts. Furthermore, developers are in a good position to set up infrastructure and write any custom tools needed.

Also, Chapter five, Programming by contract, and chapter 16, Duplication, were not bad, but they were the least interesting for me.

Odds and Ends

Some other small observations:

• The summaries at the end of each chapter are the best I have read in any technical book – they succinctly capture the key points of the chapter.
• The author introduces the term “program element” to use instead of writing “method or function” – very good.
• The illustrations are professionally done, which I prefer over amateur hand-drawings that some books use.
• The headings and sub-headings are a little too close in font size for me – it is sometimes hard to tell the levels apart.
• There are a few places where odd words or phrases are used – better editing would have helped.

Summary

Writing good software requires testing. Developer Testing gives a good introduction to the various methods and tools developers can use to test their own code. The key benefit of the book however is its advice on how to design for testability. Many testing techniques covered, like unit testing, should already be familiar to most developers. But overall it is a good overview that will help put focus on the important practice of testing your code as effectively as possible.