book

Effective Software Testing

by Mauricio Aniche

April 2022

Intermediate to advanced

328 pages

9h 56m

English

Manning Publications

Read now

Unlock full access

forewordsprefaceacknowledgmentsabout this bookWho should read this bookHow this book is organized: A roadmapWhat this book does not coverAbout the codeliveBook discussion forumabout the authorabout the cover illustration
1.1 Developers who test vs. developers who do not1.2 Effective software testing for developers1.2.1 Effective testing in the development process1.2.2 Effective testing as an iterative process1.2.3 Focusing on development and then on testing1.2.4 The myth of “correctness by design”1.2.5 The cost of testing1.2.6 The meaning of effective and systematic1.2.7 The role of test automation1.3 Principles of software testing (or, why testing is so difficult)1.3.1 Exhaustive testing is impossible1.3.2 Knowing when to stop testing1.3.3 Variability is important (the pesticide paradox)1.3.4 Bugs happen in some places more than others1.3.5 No matter what testing you do, it will never be perfect or enough1.3.6 Context is king1.3.7 Verification is not validation1.4 The testing pyramid, and where we should focus1.4.1 Unit testing1.4.2 Integration testing1.4.3 System testing1.4.4 When to use each test level1.4.5 Why do I favor unit tests?1.4.6 What do I test at the different levels?1.4.7 What if you disagree with the testing pyramid?1.4.8 Will this book help you find all the bugs?ExercisesSummary
2.1 The requirements say it all2.1.1 Step 1: Understanding the requirements, inputs, and outputs2.1.2 Step 2: Explore what the program does for various inputs2.1.3 Step 3: Explore possible inputs and outputs, and identify partitions2.1.4 Step 4: Analyze the boundaries2.1.5 Step 5: Devise test cases2.1.6 Step 6: Automate the test cases2.1.7 Step 7: Augment the test suite with creativity and experience2.2 Specification-based testing in a nutshell2.3 Finding bugs with specification testing2.4 Specification-based testing in the real world2.4.1 The process should be iterative, not sequential2.4.2 How far should specification testing go?2.4.3 Partition or boundary? It does not matter!2.4.4 On and off points are enough, but feel free to add in and out points2.4.5 Use variations of the same input to facilitate understanding2.4.6 When the number of combinations explodes, be pragmatic2.4.7 When in doubt, go for the simplest input2.4.8 Pick reasonable values for inputs you do not care about2.4.9 Test for nulls and exceptional cases, but only when it makes sense2.4.10 Go for parameterized tests when tests have the same skeleton2.4.11 Requirements can be of any granularity2.4.12 How does this work with classes and state?2.4.13 The role of experience and creativityExercisesSummary
3.1 Code coverage, the right way3.2 Structural testing in a nutshell3.3 Code coverage criteria3.3.1 Line coverage3.3.2 Branch coverage3.3.3 Condition + branch coverage3.3.4 Path coverage3.4 Complex conditions and the MC/DC coverage criterion3.4.1 An abstract example3.4.2 Creating a test suite that achieves MC/DC3.5 Handling loops and similar constructs3.6 Criteria subsumption, and choosing a criterion3.7 Specification-based and structural testing: A running example3.8 Boundary testing and structural testing3.9 Structural testing alone often is not enough3.10 Structural testing in the real world3.10.1 Why do some people hate code coverage?3.10.2 What does it mean to achieve 100% coverage?3.10.3 What coverage criterion to use3.10.4 MC/DC when expressions are too complex and cannot be simplified3.10.5 Other coverage criteria3.10.6 What should not be covered?3.11 Mutation testingExercisesSummary
4.1 Pre-conditions and post-conditions4.1.1 The assert keyword4.1.2 Strong and weak pre- and post-conditions4.2 Invariants4.3 Changing contracts, and the Liskov substitution principle4.3.1 Inheritance and contracts4.4 How is design-by-contract related to testing?4.5 Design-by-contract in the real world4.5.1 Weak or strong pre-conditions?4.5.2 Input validation, contracts, or both?4.5.3 Asserts and exceptions: When to use one or the other4.5.4 Exception or soft return values?4.5.5 When not to use design-by-contract4.5.6 Should we write tests for pre-conditions, post-conditions, and invariants?4.5.7 Tooling supportExercisesSummary

5.1 Example 1: The passing grade program5.2 Example 2: Testing the unique method5.3 Example 3: Testing the indexOf method5.4 Example 4: Testing the Basket class5.5 Example 5: Creating complex domain objects5.6 Property-based testing in the real world5.6.1 Example-based testing vs. property-based testing5.6.2 Common issues in property-based tests5.6.3 Creativity is keyExercisesSummary
6.1 Dummies, fakes, stubs, spies, and mocks6.1.1 Dummy objects6.1.2 Fake objects6.1.3 Stubs6.1.4 Mocks6.1.5 Spies6.2 An introduction to mocking frameworks6.2.1 Stubbing dependencies6.2.2 Mocks and expectations6.2.3 Capturing arguments6.2.4 Simulating exceptions6.3 Mocks in the real world6.3.1 The disadvantages of mocking6.3.2 What to mock and what not to mock6.3.3 Date and time wrappers6.3.4 Mocking types you do not own6.3.5 What do others say about mocking?ExercisesSummary
7.1 Separating infrastructure code from domain code7.2 Dependency injection and controllability7.3 Making your classes and methods observable7.3.1 Example 1: Introducing methods to facilitate assertions7.3.2 Example 2: Observing the behavior of void methods7.4 Dependency via class constructor or value via method parameter?7.5 Designing for testability in the real world7.5.1 The cohesion of the class under test7.5.2 The coupling of the class under test7.5.3 Complex conditions and testability7.5.4 Private methods and testability7.5.5 Static methods, singletons, and testability7.5.6 The Hexagonal Architecture and mocks as a design technique7.5.7 Further reading about designing for testabilityExercisesSummary
8.1 Our first TDD session8.2 Reflecting on our first TDD experience8.3 TDD in the real world8.3.1 To TDD or not to TDD?8.3.2 TDD 100% of the time?8.3.3 Does TDD work for all types of applications and domains?8.3.4 What does the research say about TDD?8.3.5 Other schools of TDD8.3.6 TDD and proper testingExercisesSummary
9.1 When to use larger tests9.1.1 Testing larger components9.1.2 Testing larger components that go beyond our code base9.2 Database and SQL testing9.2.1 What to test in a SQL query9.2.2 Writing automated tests for SQL queries9.2.3 Setting up infrastructure for SQL tests9.2.4 Best practices9.3 System tests9.3.1 An introduction to Selenium9.3.2 Designing page objects9.3.3 Patterns and best practices9.4 Final notes on larger tests9.4.1 How do all the testing techniques fit?9.4.2 Perform cost/benefit analysis9.4.3 Be careful with methods that are covered but not tested9.4.4 Proper code infrastructure is key9.4.5 DSLs and tools for stakeholders to write tests9.4.6 Testing other types of web systemsExercisesSummary
10.1 Principles of maintainable test code10.1.1 Tests should be fast10.1.2 Tests should be cohesive, independent, and isolated10.1.3 Tests should have a reason to exist10.1.4 Tests should be repeatable and not flaky10.1.5 Tests should have strong assertions10.1.6 Tests should break if the behavior changes10.1.7 Tests should have a single and clear reason to fail10.1.8 Tests should be easy to write10.1.9 Tests should be easy to read10.1.10 Tests should be easy to change and evolve10.2 Test smells10.2.1 Excessive duplication10.2.2 Unclear assertions10.2.3 Bad handling of complex or external resources10.2.4 Fixtures that are too general10.2.5 Sensitive assertionsExercisesSummary
11.1 Although the model looks linear, iterations are fundamental11.2 Bug-free software development: Reality or myth?11.3 Involve your final user11.4 Unit testing is hard in practice11.5 Invest in monitoring11.6 What’s next?
Chapter 1Chapter 2Chapter 3Chapter 4Chapter 5Chapter 6Chapter 7Chapter 8Chapter 9Chapter 10

Overview

Go beyond basic testing! Great software testing makes the entire development process more efficient. This book reveals a systemic and effective approach that will help you customize your testing coverage and catch bugs in tricky corner cases.

In Effective Software Testing you will learn how to:

Engineer tests with a much higher chance of finding bugs
Read code coverage metrics and use them to improve your test suite
Understand when to use unit tests, integration tests, and system tests
Use mocks and stubs to simplify your unit testing
Think of pre-conditions, post-conditions, invariants, and contracts
Implement property-based tests
Utilize coding practices like dependency injection and hexagonal architecture that make your software easier to test
Write good and maintainable test code

Effective Software Testing teaches you a systematic approach to software testing that will ensure the quality of your code. It’s full of techniques drawn from proven research in software engineering, and each chapter puts a new technique into practice. Follow the real-world use cases and detailed code samples, and you’ll soon be engineering tests that find bugs in edge cases and parts of code you’d never think of testing! Along the way, you’ll develop an intuition for testing that can save years of learning by trial and error.

About the Technology
Effective testing ensures that you’ll deliver quality software. For software engineers, testing is a key part of the development process. Mastering specification-based testing, boundary testing, structural testing, and other core strategies is essential to writing good tests and catching bugs before they hit production.

About the Book
Effective Software Testing is a hands-on guide to creating bug-free software. Written for developers, it guides you through all the different types of testing, from single units up to entire components. You’ll also learn how to engineer code that facilitates testing and how to write easy-to-maintain test code. Offering a thorough, systematic approach, this book includes annotated source code samples, realistic scenarios, and reasoned explanations.

What's Inside

Design rigorous test suites that actually find bugs
When to use unit tests, integration tests, and system tests
Pre-and post-conditions, invariants, contracts, and property-based tests
Design systems that are test-friendly
Test code best practices and test smells

About the Reader
The Java-based examples illustrate concepts you can use for any object-oriented language.

About the Author
Dr. Maurício Aniche is the Tech Academy Lead at Adyen and an Assistant Professor in Software Engineering at the Delft University of Technology.

Quotes
Written for the working developer, it offers state-of-the-art software testing techniques. Perfect for training the next generation of effective software testers.
- From the Foreword by Dr. Arie Van Deursen, Professor in Software Engineering, Delft University

The combination of theory and practice shows the depth of Maurício’s experience as an academic and as a working programmer!
- From the Foreword by Steve Freeman, author of Growing Object-Oriented Software, Guided by Tests

Whether you’re a developer, tester, or student, you’ll find this text an essential part of your work.
- James McKean Wood, Trimble PPM

The first book I’ve read where I finally understood how to write a good unit test, and how to pair it with testable code.
- James Liu, Mediaocean