Effective Software Testing

Effective Software Testing

by Mauricio Aniche
Released April 2022
Publisher(s): Manning Publications
ISBN: 9781633439931

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Go beyond basic testing! Great software testing makes the entire development process more efficient. This book reveals a systemic and effective approach that will help you customize your testing coverage and catch bugs in tricky corner cases.

In Effective Software Testing you will learn how to:

  • Engineer tests with a much higher chance of finding bugs
  • Read code coverage metrics and use them to improve your test suite
  • Understand when to use unit tests, integration tests, and system tests
  • Use mocks and stubs to simplify your unit testing
  • Think of pre-conditions, post-conditions, invariants, and contracts
  • Implement property-based tests
  • Utilize coding practices like dependency injection and hexagonal architecture that make your software easier to test
  • Write good and maintainable test code

Effective Software Testing teaches you a systematic approach to software testing that will ensure the quality of your code. It’s full of techniques drawn from proven research in software engineering, and each chapter puts a new technique into practice. Follow the real-world use cases and detailed code samples, and you’ll soon be engineering tests that find bugs in edge cases and parts of code you’d never think of testing! Along the way, you’ll develop an intuition for testing that can save years of learning by trial and error.

About the Technology
Effective testing ensures that you’ll deliver quality software. For software engineers, testing is a key part of the development process. Mastering specification-based testing, boundary testing, structural testing, and other core strategies is essential to writing good tests and catching bugs before they hit production.

About the Book
Effective Software Testing is a hands-on guide to creating bug-free software. Written for developers, it guides you through all the different types of testing, from single units up to entire components. You’ll also learn how to engineer code that facilitates testing and how to write easy-to-maintain test code. Offering a thorough, systematic approach, this book includes annotated source code samples, realistic scenarios, and reasoned explanations.

What's Inside
  • Design rigorous test suites that actually find bugs
  • When to use unit tests, integration tests, and system tests
  • Pre-and post-conditions, invariants, contracts, and property-based tests
  • Design systems that are test-friendly
  • Test code best practices and test smells


About the Reader
The Java-based examples illustrate concepts you can use for any object-oriented language.

About the Author
Dr. Maurício Aniche is the Tech Academy Lead at Adyen and an Assistant Professor in Software Engineering at the Delft University of Technology.

Quotes
Written for the working developer, it offers state-of-the-art software testing techniques. Perfect for training the next generation of effective software testers.
- From the Foreword by Dr. Arie Van Deursen, Professor in Software Engineering, Delft University

The combination of theory and practice shows the depth of Maurício’s experience as an academic and as a working programmer!
- From the Foreword by Steve Freeman, author of Growing Object-Oriented Software, Guided by Tests

Whether you’re a developer, tester, or student, you’ll find this text an essential part of your work.
- James McKean Wood, Trimble PPM

The first book I’ve read where I finally understood how to write a good unit test, and how to pair it with testable code.
- James Liu, Mediaocean

Publisher resources

View/Submit Errata

Table of contents

  1. inside front cover
  2. Effective Software Testing
  3. Copyright
  4. brief contents
  5. contents
  6. front matter
    1. forewords
    2. preface
    3. acknowledgments
    4. about this book
      1. Who should read this book
      2. How this book is organized: A roadmap
      3. What this book does not cover
      4. About the code
      5. liveBook discussion forum
    5. about the author
    6. about the cover illustration
  7. 1 Effective and systematic software testing
    1. 1.1 Developers who test vs. developers who do not
    2. 1.2 Effective software testing for developers
      1. 1.2.1 Effective testing in the development process
      2. 1.2.2 Effective testing as an iterative process
      3. 1.2.3 Focusing on development and then on testing
      4. 1.2.4 The myth of “correctness by design”
      5. 1.2.5 The cost of testing
      6. 1.2.6 The meaning of effective and systematic
      7. 1.2.7 The role of test automation
    3. 1.3 Principles of software testing (or, why testing is so difficult)
      1. 1.3.1 Exhaustive testing is impossible
      2. 1.3.2 Knowing when to stop testing
      3. 1.3.3 Variability is important (the pesticide paradox)
      4. 1.3.4 Bugs happen in some places more than others
      5. 1.3.5 No matter what testing you do, it will never be perfect or enough
      6. 1.3.6 Context is king
      7. 1.3.7 Verification is not validation
    4. 1.4 The testing pyramid, and where we should focus
      1. 1.4.1 Unit testing
      2. 1.4.2 Integration testing
      3. 1.4.3 System testing
      4. 1.4.4 When to use each test level
      5. 1.4.5 Why do I favor unit tests?
      6. 1.4.6 What do I test at the different levels?
      7. 1.4.7 What if you disagree with the testing pyramid?
      8. 1.4.8 Will this book help you find all the bugs?
    5. Exercises
    6. Summary
  8. 2 Specification-based testing
    1. 2.1 The requirements say it all
      1. 2.1.1 Step 1: Understanding the requirements, inputs, and outputs
      2. 2.1.2 Step 2: Explore what the program does for various inputs
      3. 2.1.3 Step 3: Explore possible inputs and outputs, and identify partitions
      4. 2.1.4 Step 4: Analyze the boundaries
      5. 2.1.5 Step 5: Devise test cases
      6. 2.1.6 Step 6: Automate the test cases
      7. 2.1.7 Step 7: Augment the test suite with creativity and experience
    2. 2.2 Specification-based testing in a nutshell
    3. 2.3 Finding bugs with specification testing
    4. 2.4 Specification-based testing in the real world
      1. 2.4.1 The process should be iterative, not sequential
      2. 2.4.2 How far should specification testing go?
      3. 2.4.3 Partition or boundary? It does not matter!
      4. 2.4.4 On and off points are enough, but feel free to add in and out points
      5. 2.4.5 Use variations of the same input to facilitate understanding
      6. 2.4.6 When the number of combinations explodes, be pragmatic
      7. 2.4.7 When in doubt, go for the simplest input
      8. 2.4.8 Pick reasonable values for inputs you do not care about
      9. 2.4.9 Test for nulls and exceptional cases, but only when it makes sense
      10. 2.4.10 Go for parameterized tests when tests have the same skeleton
      11. 2.4.11 Requirements can be of any granularity
      12. 2.4.12 How does this work with classes and state?
      13. 2.4.13 The role of experience and creativity
    5. Exercises
    6. Summary
  9. 3 Structural testing and code coverage
    1. 3.1 Code coverage, the right way
    2. 3.2 Structural testing in a nutshell
    3. 3.3 Code coverage criteria
      1. 3.3.1 Line coverage
      2. 3.3.2 Branch coverage
      3. 3.3.3 Condition + branch coverage
      4. 3.3.4 Path coverage
    4. 3.4 Complex conditions and the MC/DC coverage criterion
      1. 3.4.1 An abstract example
      2. 3.4.2 Creating a test suite that achieves MC/DC
    5. 3.5 Handling loops and similar constructs
    6. 3.6 Criteria subsumption, and choosing a criterion
    7. 3.7 Specification-based and structural testing: A running example
    8. 3.8 Boundary testing and structural testing
    9. 3.9 Structural testing alone often is not enough
    10. 3.10 Structural testing in the real world
      1. 3.10.1 Why do some people hate code coverage?
      2. 3.10.2 What does it mean to achieve 100% coverage?
      3. 3.10.3 What coverage criterion to use
      4. 3.10.4 MC/DC when expressions are too complex and cannot be simplified
      5. 3.10.5 Other coverage criteria
      6. 3.10.6 What should not be covered?
    11. 3.11 Mutation testing
    12. Exercises
    13. Summary
  10. 4 Designing contracts
    1. 4.1 Pre-conditions and post-conditions
      1. 4.1.1 The assert keyword
      2. 4.1.2 Strong and weak pre- and post-conditions
    2. 4.2 Invariants
    3. 4.3 Changing contracts, and the Liskov substitution principle
      1. 4.3.1 Inheritance and contracts
    4. 4.4 How is design-by-contract related to testing?
    5. 4.5 Design-by-contract in the real world
      1. 4.5.1 Weak or strong pre-conditions?
      2. 4.5.2 Input validation, contracts, or both?
      3. 4.5.3 Asserts and exceptions: When to use one or the other
      4. 4.5.4 Exception or soft return values?
      5. 4.5.5 When not to use design-by-contract
      6. 4.5.6 Should we write tests for pre-conditions, post-conditions, and invariants?
      7. 4.5.7 Tooling support
    6. Exercises
    7. Summary
  11. 5 Property-based testing
    1. 5.1 Example 1: The passing grade program
    2. 5.2 Example 2: Testing the unique method
    3. 5.3 Example 3: Testing the indexOf method
    4. 5.4 Example 4: Testing the Basket class
    5. 5.5 Example 5: Creating complex domain objects
    6. 5.6 Property-based testing in the real world
      1. 5.6.1 Example-based testing vs. property-based testing
      2. 5.6.2 Common issues in property-based tests
      3. 5.6.3 Creativity is key
    7. Exercises
    8. Summary
  12. 6 Test doubles and mocks
    1. 6.1 Dummies, fakes, stubs, spies, and mocks
      1. 6.1.1 Dummy objects
      2. 6.1.2 Fake objects
      3. 6.1.3 Stubs
      4. 6.1.4 Mocks
      5. 6.1.5 Spies
    2. 6.2 An introduction to mocking frameworks
      1. 6.2.1 Stubbing dependencies
      2. 6.2.2 Mocks and expectations
      3. 6.2.3 Capturing arguments
      4. 6.2.4 Simulating exceptions
    3. 6.3 Mocks in the real world
      1. 6.3.1 The disadvantages of mocking
      2. 6.3.2 What to mock and what not to mock
      3. 6.3.3 Date and time wrappers
      4. 6.3.4 Mocking types you do not own
      5. 6.3.5 What do others say about mocking?
    4. Exercises
    5. Summary
  13. 7 Designing for testability
    1. 7.1 Separating infrastructure code from domain code
    2. 7.2 Dependency injection and controllability
    3. 7.3 Making your classes and methods observable
      1. 7.3.1 Example 1: Introducing methods to facilitate assertions
      2. 7.3.2 Example 2: Observing the behavior of void methods
    4. 7.4 Dependency via class constructor or value via method parameter?
    5. 7.5 Designing for testability in the real world
      1. 7.5.1 The cohesion of the class under test
      2. 7.5.2 The coupling of the class under test
      3. 7.5.3 Complex conditions and testability
      4. 7.5.4 Private methods and testability
      5. 7.5.5 Static methods, singletons, and testability
      6. 7.5.6 The Hexagonal Architecture and mocks as a design technique
      7. 7.5.7 Further reading about designing for testability
    6. Exercises
    7. Summary
  14. 8 Test-driven development
    1. 8.1 Our first TDD session
    2. 8.2 Reflecting on our first TDD experience
    3. 8.3 TDD in the real world
      1. 8.3.1 To TDD or not to TDD?
      2. 8.3.2 TDD 100% of the time?
      3. 8.3.3 Does TDD work for all types of applications and domains?
      4. 8.3.4 What does the research say about TDD?
      5. 8.3.5 Other schools of TDD
      6. 8.3.6 TDD and proper testing
    4. Exercises
    5. Summary
  15. 9 Writing larger tests
    1. 9.1 When to use larger tests
      1. 9.1.1 Testing larger components
      2. 9.1.2 Testing larger components that go beyond our code base
    2. 9.2 Database and SQL testing
      1. 9.2.1 What to test in a SQL query
      2. 9.2.2 Writing automated tests for SQL queries
      3. 9.2.3 Setting up infrastructure for SQL tests
      4. 9.2.4 Best practices
    3. 9.3 System tests
      1. 9.3.1 An introduction to Selenium
      2. 9.3.2 Designing page objects
      3. 9.3.3 Patterns and best practices
    4. 9.4 Final notes on larger tests
      1. 9.4.1 How do all the testing techniques fit?
      2. 9.4.2 Perform cost/benefit analysis
      3. 9.4.3 Be careful with methods that are covered but not tested
      4. 9.4.4 Proper code infrastructure is key
      5. 9.4.5 DSLs and tools for stakeholders to write tests
      6. 9.4.6 Testing other types of web systems
    5. Exercises
    6. Summary
  16. 10 Test code quality
    1. 10.1 Principles of maintainable test code
      1. 10.1.1 Tests should be fast
      2. 10.1.2 Tests should be cohesive, independent, and isolated
      3. 10.1.3 Tests should have a reason to exist
      4. 10.1.4 Tests should be repeatable and not flaky
      5. 10.1.5 Tests should have strong assertions
      6. 10.1.6 Tests should break if the behavior changes
      7. 10.1.7 Tests should have a single and clear reason to fail
      8. 10.1.8 Tests should be easy to write
      9. 10.1.9 Tests should be easy to read
      10. 10.1.10 Tests should be easy to change and evolve
    2. 10.2 Test smells
      1. 10.2.1 Excessive duplication
      2. 10.2.2 Unclear assertions
      3. 10.2.3 Bad handling of complex or external resources
      4. 10.2.4 Fixtures that are too general
      5. 10.2.5 Sensitive assertions
    3. Exercises
    4. Summary
  17. 11 Wrapping up the book
    1. 11.1 Although the model looks linear, iterations are fundamental
    2. 11.2 Bug-free software development: Reality or myth?
    3. 11.3 Involve your final user
    4. 11.4 Unit testing is hard in practice
    5. 11.5 Invest in monitoring
    6. 11.6 What’s next?
  18. Appendix. Answers to exercises
    1. Chapter 1
    2. Chapter 2
    3. Chapter 3
    4. Chapter 4
    5. Chapter 5
    6. Chapter 6
    7. Chapter 7
    8. Chapter 8
    9. Chapter 9
    10. Chapter 10
  19. References
  20. index

Product information

  • Title: Effective Software Testing
  • Author(s): Mauricio Aniche
  • Release date: April 2022
  • Publisher(s): Manning Publications
  • ISBN: 9781633439931