Robust Python

Chapter 22. Acceptance Testing

As a developer, it is easy to focus on the tests that directly surround your codebase: unit tests, integration tests, UI tests, and the like. These tests verify that the code is doing what you intend. They are an invaluable tool to keep your codebase regression free. They are also completely the wrong tool for building what a customer expects.

Developers write these tests with full knowledge of the code, which means the tests are biased toward that developer’s expectations. There is no guarantee that this tested behavior is actually what the customer wants, though.

Consider the following unit test:

def test_chili_has_correct_ingredients():
    assert make_chili().ingredients() == [
        "Ground Beef",
        "Chile Blend",
        "Onion",
        ...
        "Tomatoes",
        "Pinto Beans"
    ]

This test might be airtight; it passes and catches any regression made in the code. However, when presented to a customer, you might be confronted with: “No, I wanted Texas-style chili! You know, no tomatoes or beans?” All the unit tests in the world won’t save you from building the wrong thing.

This is where acceptance testing comes in. Acceptance tests check that you are building the correct product. While unit tests and integration tests are a form of verification, acceptance tests are validation. They validate that you are building what the user expects.

In this chapter, you will learn about acceptance testing in Python. I’ll show you the behave framework, which uses the Gherkin language to define requirements ...