book

Software Testing: Principles and Practices

by Srinivasan Desikan, Gopalaswamy Ramesh

January 2007

Intermediate to advanced

503 pages

14h 27m

English

Pearson India

Read now

Unlock full access

1.1 Context of Testing in Producing Software1.2 About this Chapter1.3 The Incomplete Car1.4 Dijkstra's Doctrine1.5 A Test in Time!1.6 The Cat and the Saint1.7 Test the Tests First!1.8 The Pesticide Paradox1.9 The Convoy and the Rags1.10 The Policemen on the Bridge1.11 The Ends of the Pendulum1.12 Men in Black1.13 Automation Syndrome1.14 Putting it All TogetherReferencesProblems and Exercises
2.1 Phases of Software Project2.1.1 Requirements Gathering and Analysis2.1.2 Planning2.1.3 Design2.1.4 Development or Coding2.1.5 Testing2.1.6 Deployment and Maintenance2.2 Quality, Quality Assurance, and Quality Control2.3 Testing, Verification, and Validation2.4 Process Model to Represent Different Phases2.5 Life Cycle Models2.5.1 Waterfall Model2.5.2 Prototyping and Rapid Application Development Models2.5.3 Spiral or Iterative Model2.5.4 The V Model2.5.5 Modified V Model2.5.6 Comparison of Various Life Cycle ModelsReferencesProblems and Exercises

3.1 What is White Box Testing?3.2 Static Testing3.2.1 Static Testing by Humans3.2.2 Static Analysis Tools3.3 Structural Testing3.3.1 Unit/Code Functional Testing3.3.2 Code Coverage Testing3.3.3 Code Complexity Testing3.4 Challenges in White Box TestingReferencesProblems and Exercises
4.1 What is Black Box Testing?4.2 Why Black Box Testing?4.3 When to do Black Box Testing?4.4 How to do Black Box Testing?4.4.1 Requirements Based Testing4.4.2 Positive and Negative Testing4.4.3 Boundary Value Analysis4.4.4 Decision Tables4.4.5 Equivalence Partitioning4.4.6 State Based or Graph Based Testing4.4.7 Compatibility Testing4.4.8 User Documentation Testing4.4.9 Domain Testing4.5 ConclusionReferencesProblems and Exercises
5.1 What is Integration Testing?5.2 Integration Testing as a Type of Testing5.2.1 Top-Down Integration5.2.2 Bottom-Up Integration5.2.3 Bi-Directional Integration5.2.4 System Integration5.2.5 Choosing Integration Method5.3 Integration Testing as a Phase of Testing5.4 Scenario Testing5.4.1 System Scenarios5.4.2 Use Case Scenarios5.5 Defect Bash5.5.1 Choosing the Frequency and Duration of Defect Bash5.5.2 Selecting the Right Product Build5.5.3 Communicating the Objective of Defect Bash5.5.4 Setting up and Monitoring the Lab5.5.5 Taking Actions and Fixing Issues5.5.6 Optimizing the Effort Involved in Defect Bash5.6 ConclusionReferencesProblems and Exercises
6.1 System Testing Overview6.2 Why is System Testing Done?6.3 Functional Versus Non-Functional Testing6.4 Functional System Testing6.4.1 Design/Architecture Verification6.4.2 Business Vertical Testing6.4.3 Deployment Testing6.4.4 Beta Testing6.4.5 Certification, Standards and Testing for Compliance6.5 Non-Functional Testing6.5.1 Setting up the Configuration6.5.2 Coming up with Entry/Exit Criteria6.5.3 Balancing Key Resources6.5.4 Scalability Testing6.5.5 Reliability Testing6.5.6 Stress Testing6.5.7 Interoperability Testing6.6 Acceptance Testing6.6.1 Acceptance Criteria6.6.2 Selecting Test Cases for Acceptance Testing6.6.3 Executing Acceptance Tests6.7 Summary of Testing Phases6.7.1 Multiphase Testing Model6.7.2 Working Across Multiple Releases6.7.3 Who Does What and WhenReferencesProblems and Exercises
7.1 Introduction7.2 Factors Governing Performance Testing7.3 Methodology for Performance Testing7.3.1 Collecting Requirements7.3.2 Writing Test Cases7.3.3 Automating Performance Test Cases7.3.4 Executing Performance Test Cases7.3.5 Analyzing the Performance Test Results7.3.6 Performance Tuning7.3.7 Performance Benchmarking7.3.8 Capacity Planning7.4 Tools for Performance Testing7.5 Process for Performance Testing7.6 ChallengesReferencesProblems and Exercises
8.1 What is Regression Testing?8.2 Types of Regression Testing8.3 When to do Regression Testing?8.4 How to do Regression Testing?8.4.1 Performing an Initial "Smoke" or "Sanity" Test8.4.2 Understanding the Criteria for Selecting the Test Cases8.4.3 Classifying Test Cases8.4.4 Methodology for Selecting Test Cases8.4.5 Resetting the Test Cases for Regression Testing8.4.6 Concluding the Results of Regression Testing8.5 Best Practices in Regression TestingReferencesProblems and Exercises
9.1 Introduction9.2 Primer on Internationalization9.2.1 Definition of Language9.2.2 Character Set9.2.3 Locale9.2.4 Terms Used in This Chapter9.3 Test Phases for Internationalization Testing9.4 Enabling Testing9.5 Locale Testing9.6 Internationalization Validation9.7 Fake Language Testing9.8 Language Testing9.9 Localization Testing9.10 Tools Used for Internationalization9.11 Challenges and IssuesReferencesProblems and Exercises
10.1 Overview of Ad Hoc Testing10.2 Buddy Testing10.3 Pair Testing10.3.1 Situations When Pair Testing Becomes Ineffective10.4 Exploratory Testing10.4.1 Exploratory Testing Techniques10.5 Iterative Testing10.6 Agile and Extreme Testing10.6.1 XP Work Flow10.6.2 Summary with an Example10.7 Defect Seeding10.8 ConclusionReferencesProblems and Exercises
11.1 Introduction11.2 Primer on Object-Oriented Software11.3 Differences in OO Testing11.3.1 Unit Testing a set of Classes11.3.2 Putting Classes to Work Together—Integration Testing11.3.3 System Testing and Interoperability of OO Systems11.3.4 Regression Testing of OO Systems11.3.5 Tools for Testing of OO Systems11.3.6 SummaryReferencesProblems and Exercises
12.1 What is Usability Testing?12.2 Approach to Usability12.3 When to do Usability Testing?12.4 How to Achieve Usability?12.5 Quality Factors for Usability12.6 Aesthetics Testing12.7 Accessibility Testing12.7.1 Basic Accessibility12.7.2 Product Accessibility12.8 Tools for Usability12.9 Usability Lab Setup12.10 Test Roles for Usability12.11 SummaryReferencesProblems and Exercises
13.1 Perceptions and Misconceptions About Testing13.1.1 “Testing is not Technically Challenging”13.1.2 “Testing Does Not Provide me a Career Path or Growth”13.1.3 “I Am Put in Testing—What is Wrong With Me?!”30313.1.4 “These Folks Are My Adversaries”13.1.5 “Testing is What I Can Do in the End if I Get Time”13.1.6 “There is no Sense of Ownership in Testing”13.1.7 “Testing is only Destructive”13.2 Comparison between Testing and Development Functions13.3 Providing Career Paths for Testing Professionals13.4 The Role of the Ecosystem and a Call for Action13.4.1 Role of Education System13.4.2 Role of Senior Management13.4.3 Role of the CommunityReferencesProblems and Exercises
14.1 Dimensions of Organization Structures14.2 Structures in Single-Product Companies14.2.1 Testing Team Structures for Single-Product Companies14.2.2 Component-Wise Testing Teams14.3 Structures for Multi-Product Companies14.3.1 Testing Teams as Part of “CTO's Office”14.3.2 Single Test Team for All Products14.3.3 Testing Teams Organized by Product14.3.4 Separate Testing Teams for Different Phases of Testing14.3.5 Hybrid Models14.4 Effects of Globalization and Geographically Distributed Teams on Product Testing14.4.1 Business Impact of Globalization14.4.2 Round the Clock Development/Testing Model14.4.3 Testing Competency Center Model14.4.4 Challenges in Global Teams14.5 Testing Services Organizations14.5.1 Business Need for Testing Services14.5.2 Differences between Testing as a Service and Product—Testing Organizations14.5.3 Typical Roles and Responsibilities of Testing Services Organization14.5.4 Challenges and Issues in Testing Services Organizations14.6 Success Factors for Testing OrganizationsReferencesProblems and Exercises
15.1 Introduction15.2 Test Planning15.2.1 Preparing a Test Plan15.2.2 Scope Management: Deciding Features to be Tested/Not Tested15.2.3 Deciding Test Approach/Strategy15.2.4 Setting up Criteria for Testing15.2.5 Identifying Responsibilities, Staffing, and Training Needs15.2.6 Identifying Resource Requirements15.2.7 Identifying Test Deliverables15.2.8 Testing Tasks: Size and Effort Estimation15.2.9 Activity Breakdown and Scheduling15.2.10 Communications Management15.2.11 Risk Management15.3 Test Management15.3.1 Choice of Standards15.3.2 Test Infrastructure Management15.3.3 Test People Management15.3.4 Integrating with Product Release15.4 Test Process15.4.1 Putting Together and Baselining a Test Plan15.4.2 Test Case Specification15.4.3 Update of Traceability Matrix15.4.4 Identifying Possible Candidates for Automation15.4.5 Developing and Baselining Test Cases15.4.6 Executing Test Cases and Keeping Traceability Matrix Current15.4.7 Collecting and Analyzing Metrics15.4.8 Preparing Test Summary Report15.4.9 Recommending Product Release Criteria15.5 Test Reporting15.5.1 Recommending Product Release15.6 Best Practices15.6.1 Process Related Best Practices15.6.2 People Related Best Practices15.6.3 Technology Related Best PracticesAppendix A: Test Planning ChecklistAppendix B: Test Plan TemplateReferencesProblems and Exercises
16.1 What is Test Automation?16.2 Terms Used in Automation16.3 Skills Needed for Automation16.4 What to Automate, Scope of Automation16.4.1 Identifying the Types of Testing Amenable to Automation16.4.2 Automating Areas Less Prone to Change16.4.3 Automate Tests that Pertain to Standards16.4.4 Management Aspects in Automation16.5 Design and Architecture for Automation16.5.1 External Modules16.5.2 Scenario and Configuration File Modules16.5.3 Test Cases and Test Framework Modules16.5.4 Tools and Results Modules16.5.5 Report Generator and Reports/Metrics Modules16.6 Generic Requirements for Test Tool/Framework16.7 Process Model for Automation16.8 Selecting a Test Tool16.8.1 Criteria for Selecting Test Tools16.8.2 Steps for Tool Selection and Deployment16.9 Automation for Extreme Programming Model16.10 Challenges in Automation16.11 SummaryReferencesProblems and Exercises
17.1 What are Metrics and Measurements?17.2 Why Metrics in Testing?17.3 Types of Metrics17.4 Project Metrics17.4.1 Effort Variance (Planned vs Actual)17.4.2 Schedule Variance (Planned vs Actual)17.4.3 Effort Distribution Across Phases17.5 Progress Metrics17.5.1 Test Defect Metrics17.5.2 Development Defect Metrics17.6 Productivity Metrics17.6.1 Defects per 100 Hours of Testing17.6.2 Test Cases Executed per 100 Hours of Testing17.6.3 Test Cases Developed per 100 Hours of Testing17.6.4 Defects per 100 Test Cases17.6.5 Defects per 100 Failed Test Cases17.6.6 Test Phase Effectiveness17.6.7 Closed Defect Distribution17.7 Release metrics17.8 SummaryReferencesProblems and Exercises

Content preview from Software Testing: Principles and Practices

Chapter 17 Test Metrics and Measurements

In this chapter—

What are metrics and measurements
Why metrics in testing
Types of metrics
Project metrics
Progress metrics
Productivity metrics
Release metrics
Summary

17.1 WHAT ARE METRICS AND MEASUREMENTS?

This is the period we noticed excellent profits in the organization…and…Boss, you were on vacation that time!

All significant activities in a project need to be tracked to ensure that the project is going as per plan and to decide on any corrective actions. The measurement of key parameters is an integral part of tracking. Measurements first entail collecting a set of data. But, raw data ...