book

The Software Test Engineer's Handbook, 2nd Edition, 2nd Edition

by Graham Bath, Judy McKay

June 2014

Intermediate to advanced

560 pages

14h 14m

English

Rocky Nook

Read now

Unlock full access

Cover Page
Title Page
Copyright Page
Preface
Acknowledgements
Contents
1 Introduction
1.1 Structure of the Book1.2 Requirements for This Book1.2.1 Completeness1.2.2 Readability1.3 What Does “Advanced” Mean?1.4 What Is a Test Analyst?
2 Example Application, Marathon
2.1 Overview of Marathon2.2 General Requirements2.3 Use of the Marathon System2.4 Availability of the Marathon System2.5 Caveats about Marathon
3 Types of Systems
3.1 Introduction3.1.1 Systems of Systems3.1.2 Safety-Critical Systems3.1.3 Real-Time and Embedded Systems
4 Test Management Responsibilities for the Test Analyst
4.1 Introduction4.2 Monitoring and Controlling a Project4.2.1 Product (Quality) Risks4.2.2 Defects4.2.3 Test Cases4.2.4 Traceability4.2.5 Confidence4.3 Talking with Other Testers Wherever They Are4.4 Let’s Be Practical4.5 Exercises

5 The Test Process
5.1 Introduction to the Test Process5.2 Fitting the Process to the Life Cycle5.3 The Steps of the Test Process5.3.1 Test Planning, Monitoring, and Control5.3.1.1 Test Planning5.3.1.2 Test Monitoring and Control5.3.2 Test Analysis5.3.3 Test Design5.3.4 Test Implementation5.3.5 Test Execution5.3.6 Test Closure Activities5.4 Exercises
6 Specification-Based Testing Techniques
6.1 Introduction6.2 Individual Specification-Based Techniques6.2.1 Equivalence Partitioning6.2.2 Boundary Value Analysis (BVA)6.2.3 Decision Tables6.2.4 Cause-Effect Graphing6.2.5 State Transition Testing6.2.6 Combinatorial Testing—Pairwise and Orthogonal Arrays6.2.7 Combinatorial Testing—Classification Trees6.2.8 Use Case Testing6.2.9 User Story Testing6.2.10 Domain Analysis6.3 Selecting a Specification-Based Technique6.4 Let’s Be Practical6.5 Exercises
7 Defect-Based Testing Techniques
7.1 Introduction7.2 Taxonomies7.3 Applying the Technique7.4 Let’s Be Practical7.5 Exercises
8 Experience-Based Testing Techniques
8.1 Introduction8.2 Error Guessing8.3 Checklist-Based Testing8.4 Exploratory Testing8.5 Strengths and Weaknesses8.6 Let’s Be Practical8.7 Exercises
9 Functional Testing
9.1 Introduction9.2 Accuracy Testing9.3 Suitability Testing9.4 Interoperability Testing9.5 Let’s Be Practical9.6 Exercises
10 Usability and Accessibility Testing
10.1 Usability Testing10.1.1 Effectiveness10.1.2 Efficiency10.1.3 Satisfaction10.1.4 Subcharacteristics of Usability10.2 Accessibility Testing10.3 Test Process for Usability and Accessibility Testing10.3.1 Planning Issues10.3.2 Test Design10.3.3 Specifying Usability Tests10.4 Let’s Be Practical10.5 Exercises
11 Reviews for the Test Analyst
11.1 Introduction11.2 What Types of Work Products Can the Test Analyst Review?11.3 When Should the Test Analyst Do the Reviews?11.4 Issues11.4.1 How Do We Make Our Review Effective?11.4.2 Do We Have the Right People?11.4.3 What Should We Do with the Defects?11.4.4 But We Don’t Have Time to Do Reviews!11.5 Using Checklists for Reviews11.6 Checklist for Requirements Reviews11.7 Checklist for Use Case Reviews11.8 Checklist for Usability Reviews11.9 Checklist for User Story Reviews11.10 Checklist for Success11.11 Let’s Be Practical11.12 Exercises
12 Defect Management
12.1 Introduction12.2 What Is a Defect?12.3 When Can We Find Defects?12.4 Defect Fields12.4.1 Classification Information for Defects12.5 Defect Life Cycles12.6 Metrics and Reporting12.6.1 Test Progress Monitoring12.6.2 Defect Density Analysis12.6.3 Found vs. Fixed Metrics12.6.4 Convergence Metrics12.6.5 Phase Containment Information12.6.6 Is Our Defect Information Objective?12.7 Process Improvement Opportunities12.8 Let’s Be Practical12.9 Exercises
13 Tools Concepts
13.1 What Is a Test Tool?13.2 Why Would We Use a Tool?13.3 Types of Tools13.3.1 Test Design Tools13.3.2 Data Tools13.3.3 Test Execution Tools13.3.4 When Should We Automate?13.3.5 Things to Know About Automation13.3.6 Implementing Automation13.3.6.1 Data-Driven Automation13.3.6.2 Keyword-Driven Automation13.3.6.3 Benefits of Automation Techniques13.4 Should We Automate All Our Testing?13.5 Let’s Be Practical13.6 Exercises
14 Test Management Responsibilities for the Technical Test Analyst
14.1 Introduction14.1.1 Technical Risks14.2 Let’s Be Practical14.3 Exercises
15 Analysis Techniques
15.1 Static Analysis15.1.1 Benefits15.1.2 Limitations15.1.3 Control Flow Analysis15.1.4 Data Flow Analysis15.1.5 Compliance to Coding Standards15.1.6 Generating Code Metrics15.1.7 Static Analysis of a Website15.1.8 Call Graphs15.2 Dynamic Analysis15.2.1 Benefits15.2.2 Limitations15.2.3 Memory Leaks15.2.4 Problems with Pointers15.2.5 Analysis of Performance15.3 Let’s Be Practical15.4 Exercises
16 Structure-Based Testing Techniques
16.1 Benefits16.2 Drawbacks16.3 Application of Structure-Based Techniques16.4 Individual Structural Techniques16.4.1 Statement Testing16.4.2 Decision Branch Testing16.4.3 Condition Testing16.4.4 Decision Condition Testing16.4.5 Multiple Condition Testing16.4.6 Modified Condition/Decision Coverage (MC/DC) Testing16.4.7 Path Testing16.4.8 API Testing16.5 Selecting a Structure-Based Technique16.6 Exercises
17 Efficiency Testing
17.1 Overview17.2 Performance Testing17.3 Load Testing17.4 Stress Testing17.5 Scalability Testing17.6 Resource Utilization Testing17.7 Measuring Efficiency17.8 Planning of Efficiency Tests17.8.1 Risks and Typical Efficiency Defects17.8.2 Different Types of Test Objects17.8.3 Requirements for Efficiency Tests17.8.4 Approaches to Efficiency Tests17.8.5 Efficiency Pass/Fail Criteria17.8.6 Tooling for Efficiency Tests17.8.7 Environments17.8.8 Organizational Issues17.8.9 Life Cycle Issues17.9 Specifying Efficiency Tests17.10 Executing Efficiency Tests17.11 Reporting Results of Efficiency Tests17.12 Tools for Performance Testing17.13 Let’s Be Practical17.14 Exercises
18 Security Testing
18.1 Overview of Security Testing18.2 Defining Security18.3 Typical Security Threats18.4 Approach to Security Testing18.5 Organizational Issues18.6 Life Cycle Issues18.7 Planning Security Tests18.8 Security Test Analysis and Design18.8.1 Software Attacks18.8.2 Other Design Techniques for Security Tests18.9 Execution of Security Tests18.10 Reporting Security Tests18.11 Tools for Security Testing18.12 Let’s Be Practical18.13 Exercises
19 Reliability Testing
19.1 Overview19.1.1 Maturity19.1.2 Fault Tolerance19.1.3 Recoverability19.2 Reliability Test Planning19.2.1 Assessing the Risk19.2.2 Setting Reliability Goals19.2.3 Life Cycle Issues19.2.4 Approaches to Reliability Testing19.2.5 Approach for Measuring Reliability Levels19.2.6 Approach for Establishing Fault Tolerance19.2.7 Approach to Failover Testing19.2.8 Approach to Backup and Restore Testing19.3 Reliability Test Specification19.3.1 Test Specification for Reliability Growth19.3.2 Test Specification for Fault Tolerance19.3.3 Test Specification for Failover19.3.4 Test Specification for Backup and Restore19.4 Reliability Test Execution19.5 Reporting Reliability Tests19.6 Tools for Reliability Testing19.7 Let’s Be Practical19.8 Exercises
20 Maintainability Testing
20.1 Overview20.2 Testing for Maintainability20.2.1 What Is Maintainability?20.2.2 Why Is Maintainability Underrepresented?20.2.3 The Causes of Poor Maintainability20.3 Maintainability Test Planning20.4 Maintainability Test Specification20.5 Performing Maintainability Tests and Analysis20.6 Maintenance Testing20.7 Tasks of the Technical Test Analyst20.8 Let’s Be Practical20.9 Exercises
21 Portability Testing
21.1 Adaptability21.1.1 Reasons for Poor Adaptability Characteristics21.1.2 Adaptability Testing21.2 Replaceability21.2.1 Replaceability Considerations21.2.2 Replaceability Testing21.3 Installability21.3.1 Risk Factors for Installability21.3.2 Installability Testing21.4 Co-existence/compatibility21.4.1 Co-existence Testing21.5 Let’s Be Practical21.6 Exercises
22 Reviews for the Technical Test Analyst
22.1 Introduction22.2 Checklists for Reviews22.3 Checklist for Code Reviews22.4 Checklist for Architectural Reviews22.5 Exercises
23 Tools for the Technical Test Analyst
23.1 Introduction23.2 Tasks and Skills of the Technical Test Analyst in Test Automation23.3 Integration and Information Interchange between Tools23.4 Defining the Test Automation Project23.4.1 Approaches to Test Automation23.5 Should We Automate All Our Testing?23.6 Types of Tools23.6.1 Fault Seeding and Fault Injection Tools23.6.2 Test Tools for Component Testing and Build23.6.3 Tools for Static Analysis of a Website23.6.4 Tools to Support Model-Based Testing23.6.5 Test Tools for Static and Dynamic Analysis23.6.6 Performance Test Tools23.6.7 Simulation and Emulation Tools23.6.8 Debugging and Troubleshooting Tools23.7 Exercises
A Glossary
Test Analyst GlossaryTechnical Test Analyst Glossary (ATT)
B Literature
BooksISTQB PublicationsStandardsWWW Pages
Index

Content preview from The Software Test Engineer's Handbook, 2nd Edition, 2nd Edition

20 Maintainability Testing

Poor old maintainability, always relegated toward the end of the list of software characteristics, often neglected entirely in master test plans, and frequently not even recognized as the root cause when we later get bitten by symptoms of poor maintainability. You would think that more attention would be paid to this aspect of software quality, wouldn’t you? After all, there is evidence that maintenance-related tasks can account for up to 80 percent of the effort spent on an application, measured over its entire life cycle. In fact, one should expect that the vast majority of the software’s life cycle is spent in the maintenance phase. This chapter tries to redress the imbalance and make the technical test analyst more ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Advanced Software Testing - Vol. 3, 2nd Edition, 2nd Edition

Publisher Resources

ISBN: 9781492014706

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

The Software Test Engineer's Handbook, 2nd Edition, 2nd Edition

by Graham Bath, Judy McKay

20 Maintainability Testing

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.