book

Effective Software Testing

by Mauricio Aniche

April 2022

Intermediate to advanced

328 pages

9h 56m

English

Manning Publications

Read now

Unlock full access

forewordsprefaceacknowledgmentsabout this bookWho should read this bookHow this book is organized: A roadmapWhat this book does not coverAbout the codeliveBook discussion forumabout the authorabout the cover illustration
1.1 Developers who test vs. developers who do not1.2 Effective software testing for developers1.2.1 Effective testing in the development process1.2.2 Effective testing as an iterative process1.2.3 Focusing on development and then on testing1.2.4 The myth of “correctness by design”1.2.5 The cost of testing1.2.6 The meaning of effective and systematic1.2.7 The role of test automation1.3 Principles of software testing (or, why testing is so difficult)1.3.1 Exhaustive testing is impossible1.3.2 Knowing when to stop testing1.3.3 Variability is important (the pesticide paradox)1.3.4 Bugs happen in some places more than others1.3.5 No matter what testing you do, it will never be perfect or enough1.3.6 Context is king1.3.7 Verification is not validation1.4 The testing pyramid, and where we should focus1.4.1 Unit testing1.4.2 Integration testing1.4.3 System testing1.4.4 When to use each test level1.4.5 Why do I favor unit tests?1.4.6 What do I test at the different levels?1.4.7 What if you disagree with the testing pyramid?1.4.8 Will this book help you find all the bugs?ExercisesSummary
2.1 The requirements say it all2.1.1 Step 1: Understanding the requirements, inputs, and outputs2.1.2 Step 2: Explore what the program does for various inputs2.1.3 Step 3: Explore possible inputs and outputs, and identify partitions2.1.4 Step 4: Analyze the boundaries2.1.5 Step 5: Devise test cases2.1.6 Step 6: Automate the test cases2.1.7 Step 7: Augment the test suite with creativity and experience2.2 Specification-based testing in a nutshell2.3 Finding bugs with specification testing2.4 Specification-based testing in the real world2.4.1 The process should be iterative, not sequential2.4.2 How far should specification testing go?2.4.3 Partition or boundary? It does not matter!2.4.4 On and off points are enough, but feel free to add in and out points2.4.5 Use variations of the same input to facilitate understanding2.4.6 When the number of combinations explodes, be pragmatic2.4.7 When in doubt, go for the simplest input2.4.8 Pick reasonable values for inputs you do not care about2.4.9 Test for nulls and exceptional cases, but only when it makes sense2.4.10 Go for parameterized tests when tests have the same skeleton2.4.11 Requirements can be of any granularity2.4.12 How does this work with classes and state?2.4.13 The role of experience and creativityExercisesSummary
3.1 Code coverage, the right way3.2 Structural testing in a nutshell3.3 Code coverage criteria3.3.1 Line coverage3.3.2 Branch coverage3.3.3 Condition + branch coverage3.3.4 Path coverage3.4 Complex conditions and the MC/DC coverage criterion3.4.1 An abstract example3.4.2 Creating a test suite that achieves MC/DC3.5 Handling loops and similar constructs3.6 Criteria subsumption, and choosing a criterion3.7 Specification-based and structural testing: A running example3.8 Boundary testing and structural testing3.9 Structural testing alone often is not enough3.10 Structural testing in the real world3.10.1 Why do some people hate code coverage?3.10.2 What does it mean to achieve 100% coverage?3.10.3 What coverage criterion to use3.10.4 MC/DC when expressions are too complex and cannot be simplified3.10.5 Other coverage criteria3.10.6 What should not be covered?3.11 Mutation testingExercisesSummary
4.1 Pre-conditions and post-conditions4.1.1 The assert keyword4.1.2 Strong and weak pre- and post-conditions4.2 Invariants4.3 Changing contracts, and the Liskov substitution principle4.3.1 Inheritance and contracts4.4 How is design-by-contract related to testing?4.5 Design-by-contract in the real world4.5.1 Weak or strong pre-conditions?4.5.2 Input validation, contracts, or both?4.5.3 Asserts and exceptions: When to use one or the other4.5.4 Exception or soft return values?4.5.5 When not to use design-by-contract4.5.6 Should we write tests for pre-conditions, post-conditions, and invariants?4.5.7 Tooling supportExercisesSummary

5.1 Example 1: The passing grade program5.2 Example 2: Testing the unique method5.3 Example 3: Testing the indexOf method5.4 Example 4: Testing the Basket class5.5 Example 5: Creating complex domain objects5.6 Property-based testing in the real world5.6.1 Example-based testing vs. property-based testing5.6.2 Common issues in property-based tests5.6.3 Creativity is keyExercisesSummary
6.1 Dummies, fakes, stubs, spies, and mocks6.1.1 Dummy objects6.1.2 Fake objects6.1.3 Stubs6.1.4 Mocks6.1.5 Spies6.2 An introduction to mocking frameworks6.2.1 Stubbing dependencies6.2.2 Mocks and expectations6.2.3 Capturing arguments6.2.4 Simulating exceptions6.3 Mocks in the real world6.3.1 The disadvantages of mocking6.3.2 What to mock and what not to mock6.3.3 Date and time wrappers6.3.4 Mocking types you do not own6.3.5 What do others say about mocking?ExercisesSummary
7.1 Separating infrastructure code from domain code7.2 Dependency injection and controllability7.3 Making your classes and methods observable7.3.1 Example 1: Introducing methods to facilitate assertions7.3.2 Example 2: Observing the behavior of void methods7.4 Dependency via class constructor or value via method parameter?7.5 Designing for testability in the real world7.5.1 The cohesion of the class under test7.5.2 The coupling of the class under test7.5.3 Complex conditions and testability7.5.4 Private methods and testability7.5.5 Static methods, singletons, and testability7.5.6 The Hexagonal Architecture and mocks as a design technique7.5.7 Further reading about designing for testabilityExercisesSummary
8.1 Our first TDD session8.2 Reflecting on our first TDD experience8.3 TDD in the real world8.3.1 To TDD or not to TDD?8.3.2 TDD 100% of the time?8.3.3 Does TDD work for all types of applications and domains?8.3.4 What does the research say about TDD?8.3.5 Other schools of TDD8.3.6 TDD and proper testingExercisesSummary
9.1 When to use larger tests9.1.1 Testing larger components9.1.2 Testing larger components that go beyond our code base9.2 Database and SQL testing9.2.1 What to test in a SQL query9.2.2 Writing automated tests for SQL queries9.2.3 Setting up infrastructure for SQL tests9.2.4 Best practices9.3 System tests9.3.1 An introduction to Selenium9.3.2 Designing page objects9.3.3 Patterns and best practices9.4 Final notes on larger tests9.4.1 How do all the testing techniques fit?9.4.2 Perform cost/benefit analysis9.4.3 Be careful with methods that are covered but not tested9.4.4 Proper code infrastructure is key9.4.5 DSLs and tools for stakeholders to write tests9.4.6 Testing other types of web systemsExercisesSummary
10.1 Principles of maintainable test code10.1.1 Tests should be fast10.1.2 Tests should be cohesive, independent, and isolated10.1.3 Tests should have a reason to exist10.1.4 Tests should be repeatable and not flaky10.1.5 Tests should have strong assertions10.1.6 Tests should break if the behavior changes10.1.7 Tests should have a single and clear reason to fail10.1.8 Tests should be easy to write10.1.9 Tests should be easy to read10.1.10 Tests should be easy to change and evolve10.2 Test smells10.2.1 Excessive duplication10.2.2 Unclear assertions10.2.3 Bad handling of complex or external resources10.2.4 Fixtures that are too general10.2.5 Sensitive assertionsExercisesSummary
11.1 Although the model looks linear, iterations are fundamental11.2 Bug-free software development: Reality or myth?11.3 Involve your final user11.4 Unit testing is hard in practice11.5 Invest in monitoring11.6 What’s next?
Chapter 1Chapter 2Chapter 3Chapter 4Chapter 5Chapter 6Chapter 7Chapter 8Chapter 9Chapter 10

Content preview from Effective Software Testing

References

Aniche, Maurício, and Marco Aurélio Gerosa. 2015. “Does Test-Driven Development Improve Class Design? A Qualitative Study on Developers’ Perceptions.” Journal of the Brazilian Computer Society 21 (1): 1–11.

Aniche, Maurício, Eduardo Guerra, and Marco Aurélio Gerosa. 2014. “Improving Code Quality on Automated Tests of Web Applications: A Set of Patterns.” In 21st Conference on Pattern Languages of Programs (PLOP).

Aniche, Maurício, Christoph Treude, and Andy Zaidman. 2021. “How Developers Engineer Test Cases: An Observational Study.” Transactions on Software Engineering (TSE). https://arxiv.org/abs/2103.01783.

Arguelles, Carlos, Marko Ivankovic, and Adam Bender. 2020. “Google Testing Blog: Code Coverage Best Practices.” https://testing.googleblog.com/2020/08/code-coverage-best-practices.html ...