Robust Python

Chapter 21. Testing Strategy

Tests are one of most important safety nets you can build around your codebase. It is incredibly comforting to make a change and see that all tests pass afterwards. However, it is challenging to gauge the best use of your time regarding testing. Too many tests and they become a burden; you spend more time maintaining tests than delivering features. Too few tests and you are letting potential catastrophes make it into production.

In this chapter, I will ask you to focus on your testing strategy. I’ll break down the different types of tests and how to choose which tests to write. I’ll focus on Python best practices around test construction, and then I’ll end with some common testing strategies specific to Python.

Defining Your Test Strategy

Before you write tests, you should decide what your test strategy will be. A test strategy is a plan for spending time and effort to test your software in order to mitigate risk. This strategy will influence what types of tests you write, how you write them, and how much time you spend writing (and maintaining) them. Everybody’s test strategy will be different, but they will all be in a similar form: a list of questions about your system and how you plan on answering them. For example, if I were writing a calorie-counting app, here would be a part of my test strategy:

Does my system function as expected? Tests to write (automated - run daily): Acceptance tests: Adding calories to the daily count Acceptance tests: ...