Robust Python

Chapter 23. Property-Based Testing

It is impossible to test absolutely everything in your codebase. The best you can do is be smart in how you target specific use cases. You look for boundary cases, paths through the code, and any other interesting attributes of the code. Your main hope is that you haven’t left any big holes in your safety net. However, you can do better than hope. You can fill in those gaps with property-based testing.

In this chapter, you will learn how to do property-based testing with a Python library called Hypothesis. You’ll use Hypothesis to generate test cases for you, often in ways you could never expect. You’ll learn how to track failing test cases, craft input data in new ways, and even have Hypothesis create combinations of algorithms to test your software. Hypothesis will guard your codebase against a whole new combination of errors.

Property-Based Testing with Hypothesis

Property-based testing is a form of generative testing, where tools generate test cases for you. Instead of writing test cases based on specific input/output combinations, you define properties for your system. Properties in this context is another name for the invariants (discussed in Chapter 10) that hold true for your system.

Consider a menu recommendation system that selects dishes based on customer-provided constraints, such as total calories, price, and cuisine. For this specific example, I want customers to be able to order a full meal that falls below a specific calorie ...