book

Foundations of Software Testing, 2nd Edition

by Aditya P. Mathur

May 2013

Intermediate to advanced

728 pages

19h 29m

English

Pearson India

Read now

Unlock full access

1.1. Humans, Errors, and Testing1.1.1 Errors, faults, and failures1.1.2 Test automation1.1.3 Developer and tester as two roles1.2. Software Quality1.2.1 Quality attributes1.2.2 Reliability1.3. Requirements, Behavior and Correctness1.3.1 Input domain1.3.2 Specifying program behavior1.3.3 Valid and invalid inputs1.4. Correctness Versus Reliability1.4.1 Correctness1.4.2 Reliability1.4.3 Operational profiles1.5. ΤEsting and Debugging1.5.1 Preparing a test plan1.5.2 Constructing test data1.5.3 Executing the program1.5.4 Assessing program correctness1.5.5 Constructing an oracle1.6. Test Metrics1.6.1 Organizational metrics1.6.2 Project metrics1.6.3 Process metrics1.6.4 Product metrics: generic1.6.5 Product metrics: OO software1.6.6 Progress monitoring and trends1.6.7 Static and dynamic metrics1.6.8 Testability1.7. Software and Hardware Testing1.8. Testing and Verification1.9. Defect Management1.10. Test Generation Strategies1.11. Static Testing1.11.1 Walkthroughs1.11.2 Inspections1.11.3 Software complexity and static testing 1.12. Model-Based Testing and Model Checking1.13. Types of Testing1.13.1 Classifier: CI: Source of test generation1.13.2 Classifier: C2: Life cycle phase1.13.3 Classifier: C3: Goal-directed testing 1.13.4 Classifier: C4: Artifact under test1.13.5 Classifier: C5: Test process models1.14. The Saturation Effect1.14.1 Confidence and true reliability1.14.2 Saturation region1.14.3 False sense of confidence1.14.4 Reducing ∆1.14.5 Impact on test process1.15. Principles of Testing1.16. ToolsSummaryExercises
2.1. Predicates and Boolean Expressions2.2. Control Flow Graph2.2.1 Basic blocks2.2.2 Flow graphs2.2.3 Paths2.2.4 Basis paths2.2.5 Path conditions and domains2.2.6 Domain and computation errors2.2.7 Static code analysis tools and static testing2.3. Execution History2.4. Dominators and Post-dominators2.5. Program Dependence Graph2.5.1 Data dependence2.5.2 Control dependence2.5.3 Call graph2.6. Strings, Languages, and Regular Expressions2.7. ToolsSummaryExercises

3.1. Introduction3.2. The Test Selection Problem3.3. Equivalence Partitioning3.3.1 Faults targeted3.3.2 Relations3.3.3 Equivalence classes for variables3.3.4 Unidimensional partitioning versus multidimensional partitioning3.3.5 A systematic procedure3.3.6 Test selection3.3.7 Impact of GUI design3.4. Boundary Value Analysis3.5. Category-Partition Method3.5.1 Steps in the category-partition methodSummaryExercises
4.1. Introduction4.2. Domain Testing4.2.1 Domain errors4.2.2 Border shifts4.2.3 ON-OFF points4.2.4 Undetected errors4.2.5 Coincidental correctness4.2.6 Paths to be tested4.3. Cause-Effect Graphing4.3.1 Notation used in cause-effect graphing4.3.2 Creating cause-effect graphs4.3.3 Decision table from cause-effect graph4.3.4 Heuristics to avoid combinatorial explosion4.3.5 Test generation from a decision table4.4. Tests Using Predicate Syntax4.4.1 A fault model4.4.2 Missing or extra Boolean variable faults4.4.3 Predicate constraints4.4.4 Predicate testing criteria4.4.5 BOR, BRO, and BRE adequate tests4.4.6 BOR constraints for non-singular expressions4.4.7 Cause-effect graphs and predicate testing4.4.8 Fault propagation4.4.9 Predicate testing in practice4.5. Tests Using Basis Paths4.6. Scenarios and TestsSummaryExercises
5.1. Software Design and Testing5.2. Finite State Machines5.2.1 Excitation using an input sequence5.2.2 Tabular representation5.2.3 Properties of FSM5.3. Conformance Testing5.3.1 Reset inputs5.3.2 The testing problem5.4. A Fault Model5.4.1 Mutants of FSMs5.4.2 Fault coverage
5.5.1 Construction of the k -equivalence partitions5.5.2 Deriving the characterization set5.5.3 Identification sets5.6. The W-Method5.6.1 Assumptions5.6.2 Maximum number of states5.6.3 Computation of the transition cover set5.6.4 Constructing Ζ5.6.5 Deriving a test set5.6.6 Testing using the W-method5.6.7 The error detection process5.7. The Partial W-Method5.7.1 Testing using the Wp-method for m = n5.7.2 Testing using the Wp-method for m > n5.8. The UIO-Sequence Method5.8.1 Assumptions5.8.2 UIO sequences5.8.3 Core and non-core behavior5.8.4 Generation of UIO sequences5.8.5 Explanation of gen-uio5.8.6 Distinguishing signatures5.8.7 Test generation5.8.8 Test optimization5.8.9 Fault detection5.9. Automata Theoretic Versus Control-Flow Based Techniques5.9.1 n-switch-cover5.9.2 Comparing automata theoretic methods5.10. Tools SummaryExercises
6.1. Combinatorial Designs6.1.1 Test configuration and test set6.1.2 Modeling the input and configuration spaces6.2. A Combinatorial Test Design Process6.3. Fault Model6.3.1 Fault vectors6.4. Latin Squares6.5. Mutually Orthogonal Latin Squares6.6. Pairwise Design: Binary Factors6.7. Pairwise Design: Multi-valued Factors6.7.1 Shortcomings of using MOLS for test design6.8. Orthogonal Arrays6.8.1 Mixed-level orthogonal arrays6.9. Covering and Mixed-level Covering Arrays6.9.1 Mixed-level covering arrays6.10. Arrays of Strength > 26.11. Generating Covering Arrays6.12. Tools SummaryExercises
7.1. Test Adequacy: Basics7.1.1 What is test adequacy?7.1.2 Measurement of test adequacy7.1.3 Test enhancement using measurements of adequacy7.1.4 Infeasibility and test adequacy7.1.5 Error detection and test enhancement7.1.6 Single and multiple executions7.2. Adequacy Criteria Based on Control Flow7.2.1 Statement and block coverage7.2.2 Conditions and decisions7.2.3 Decision coverage7.2.4 Condition coverage7.2.5 Condition/decision coverage7.2.6 Multiple condition coverage7.2.7 Linear code sequence and jump (LCSAJ) coverage7.2.8 Modified condition/decision coverage7.2.9 MC/ DC adequate tests for compound conditions7.2.10 Definition of MC/ DC coverage7.2.11 Minimal MC/ DC tests7.2.12 Error detection and MC/DC adequacy7.2.13 Short-circuit evaluation and infeasibility7.2.14 Basis path coverage7.2.15 Tracing test cases to requirements7.3. Concepts From Data Flow7.3.1 Definitions and uses7.3.2 C-use and p-use7.3.3 Global and local definitions and uses7.3.4 Dataflow graph7.3.5 Def-clear paths7.3.6 Def-use pairs7.3.7 Def-use chains7.3.8 A little optimization7.3.9 Data contexts and ordered data contexts
7.4.1 c-use coverage7.4.2 p-use coverage7.4.3 All-uses coverage7.4.4 k-dr chain coverage7.4.5 Using the k-dr chain coverage7.4.6 Infeasible c- and p-uses7.4.7 Context coverage7.5. Control Flow Versus Data Flow7.6. The “Subsumes” Relation7.7. Structural and Functional Testing7.8. Scalability of Coverage Measurement7.9. Tools SummaryExercises
8.1. Introduction8.2. Mutation and Mutants8.2.1 First-order and higher order mutants8.2.2 Syntax and semantics of mutants8.2.3 Strong and weak mutations8.2.4 Why mutate?8.3. Test Assessment Using Mutation8.3.1 A procedure for test adequacy assessment8.3.2 Alternate procedures for test adequacy assessment8.3.3 “Distinguished” versus “killed” mutants8.3.4 Conditions for distinguishing a mutant8.4. Mutation Operators8.4.1 Operator types8.4.2 Language dependence of mutation operators8.5. Design of Mutation Operators8.5.1 Goodness criteria for mutation operators8.5.2 Guidelines
8.6.1 The competent programmer hypothesis8.6.2 The coupling effect8.7. Equivalent Mutants8.8. Fault Detection Using Mutation8.9. Types of Mutants8.10. Mutation Operators For C8.10.1 What is not mutated?8.10.2 Linearization8.10.3 Execution sequence8.10.4 Effect of an execution sequence8.10.5 Global and local identifier sets8.10.6 Global and local reference sets8.10.7 Mutating program constants8.10.8 Mutating operators8.10.9 Binary operator mutations8.10.10 Mutating statements8.10.11 Mutating program variables8.10.12 Structure Reference Replacement8.11. Mutation Operators For Java8.11.1 Traditional mutation operators8.11.2 Inheritence8.11.3 Polymorphism and dynamic binding8.11.4 Method overloading8.11.5 Java specific mutation operators8.12. Comparison of Mutation Operators8.13. Mutation Testing Within Budget8.13.1 Prioritizing functions to be mutated8.13.2 Selecting a subset of mutation operators8.14. Case and Program Testing8.15. ToolsSummaryExercises
9.1. What is Regression Testing?9.2. Regression Test Process9.2.1 Revalidation, selection, minimization, and prioritization9.2.2 Test setup9.2.3 Test sequencing9.2.4 Test execution9.2.5 Output comparison9.3. Regression Test Selection: The Problem9.4. Selecting Regression Tests9.4.1 Test all9.4.2 Random selection9.4.3 Selecting modification traversing tests9.4.4 Test minimization9.4.5 Test prioritization9.5. Test Selection Using Execution Trace9.5.1 Obtaining the execution trace9.5.2 Selecting regression tests9.5.3 Handling function calls9.5.4 Handling changes in declarations9.6. Test Selection Using Dynamic Slicing9.6.1 Dynamic slicing9.6.2 Computation of dynamic slices9.6.3 Selecting tests9.6.4 Potential dependence9.6.5 Computing the relevant slice9.6.6 Addition and deletion of statements9.6.7 Iden tifying variables for slicing9.6.8 Reduced dynamic dependence graph9.7. Scalability of Test Selection Algorithms9.8. Test Minimization9.8.1 The set cover problem9.8.2 A procedure for test minimization9.9. Test Prioritization9.10. ToolsSummaryExercises
10.1. Introduction10.2. Context10.3. Test Design10.4. Using Junit10.5. Stubs and Mocks10.5.1 Using mock objects10.6. ToolsSummaryExercises
11.1. Introduction11.2. Integration Errors11.3. Dependence11.3.1 Class relationships: static11.3.2 Class relationships: dynamic11.3.3 Class firewalls11.3.4 Precise and imprecise relationships11.4. OO versus Non-OO Programs11.5. Integration Hierarchy11.5.1 Choosing an integration strategy11.5.2 Comparing integration strategies11.5.3 Specific stubs and retesting11.6. Finding A Near-optimal Test Order11.6.1 The TD method11.6.2 The TJJM method11.6.3 The BLW method11.6.4 Comparison of TD, TJJM, and the BLW methods11.6.5 Which test order algorithm to select?11.7. Test Generation11.7.1 Data variety11.7.2 Data constraints11.8. Test Assessment11.9. ToolsSummaryExercises

Content preview from Foundations of Software Testing, 2nd Edition

Part III

Test Adequacy Assessment and Enhancement

Techniques to answer “Is my testing adequate?” are introduced in the next two chapters. Chapter 7 presents the foundations of test completeness as defined by Goodenough and Gerhart. This is followed by definitions and illustrations of test adequacy criteria based on the control flow and data flow structure of the program under test.

Chapter 8 is an in-depth presentation of some of the most powerful test adequacy criteria based on program mutation. In each of the two chapters mentioned, we provide several examples to show how errors are detected, or not detected, while enhancing an inadequate test set with reference to an adequacy criterion.