O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Software Testing, Second Edition

Software Testing, Second Edition

by Ron Patton
Publisher: Sams
Release Date: July 2005
ISBN: 0672327988
View table of contents
Start reading

Book Description

Software Testing, Second Edition provides practical insight into the world of software testing and quality assurance. Learn how to find problems in any computer program, how to plan an effective test approach and how to tell when software is ready for release. Updated from the previous edition in 2000 to include a chapter that specifically deals with testing software for security bugs, the processes and techniques used throughout the book are timeless. This book is an excellent investment if you want to better understand what your Software Test team does or you want to write better software.

Table of Contents

  1. Copyright
    1. Dedication
  2. About the Author
  3. Acknowledgments
  4. We Want to Hear from You!
  5. Introduction
    1. About the Second Edition
    2. Who Should Use This Book?
    3. What This Book Will Do for You
    4. Software Necessary to Use This Book
    5. How This Book Is Organized
      1. Part I: The Big Picture
      2. Part II: Testing Fundamentals
      3. Part III: Applying Your Testing Skills
      4. Part IV: Supplementing Your Testing
      5. Part V: Working with Test Documentation
      6. Part VI: The Future
      7. Appendix
    6. Conventions Used in This Book
  6. I. The Big Picture
    1. 1. Software Testing Background
      1. Infamous Software Error Case Studies
        1. Disney's Lion King, 1994–1995
        2. Intel Pentium Floating-Point Division Bug, 1994
        3. NASA Mars Polar Lander, 1999
        4. Patriot Missile Defense System, 1991
        5. The Y2K (Year 2000) Bug, circa 1974
        6. Dangerous Viewing Ahead, 2004
      2. What Is a Bug?
        1. Terms for Software Failures
        2. Software Bug: A Formal Definition
      3. Why Do Bugs Occur?
      4. The Cost of Bugs
      5. What Exactly Does a Software Tester Do?
      6. What Makes a Good Software Tester?
      7. Summary
      8. Quiz
    2. 2. The Software Development Process
      1. Product Components
        1. What Effort Goes Into a Software Product?
          1. Customer Requirements
          2. Specifications
          3. Schedules
          4. Software Design Documents
          5. Test Documents
        2. What Parts Make Up a Software Product?
      2. Software Project Staff
      3. Software Development Lifecycle Models
        1. Big-Bang Model
        2. Code-and-Fix Model
        3. Waterfall Model
        4. Spiral Model
      4. Summary
      5. Quiz
    3. 3. The Realities of Software Testing
      1. Testing Axioms
        1. It's Impossible to Test a Program Completely
        2. Software Testing Is a Risk-Based Exercise
        3. Testing Can't Show That Bugs Don't Exist
        4. The More Bugs You Find, the More Bugs There Are
        5. The Pesticide Paradox
        6. Not All the Bugs You Find Will Be Fixed
        7. When a Bug's a Bug Is Difficult to Say
        8. Product Specifications Are Never Final
        9. Software Testers Aren't the Most Popular Members of a Project Team
        10. Software Testing Is a Disciplined Technical Profession
      2. Software Testing Terms and Definitions
        1. Precision and Accuracy
        2. Verification and Validation
        3. Quality and Reliability
        4. Testing and Quality Assurance (QA)
      3. Summary
      4. Quiz
  7. II. Testing Fundamentals
    1. 4. Examining the Specification
      1. Getting Started
        1. Black-Box and White-Box Testing
        2. Static and Dynamic Testing
        3. Static Black-Box Testing: Testing the Specification
      2. Performing a High-Level Review of the Specification
        1. Pretend to Be the Customer
        2. Research Existing Standards and Guidelines
        3. Review and Test Similar Software
      3. Low-Level Specification Test Techniques
        1. Specification Attributes Checklist
        2. Specification Terminology Checklist
      4. Summary
      5. Quiz
    2. 5. Testing the Software with Blinders On
      1. Dynamic Black-Box Testing: Testing the Software While Blindfolded
      2. Test-to-Pass and Test-to-Fail
      3. Equivalence Partitioning
      4. Data Testing
        1. Boundary Conditions
          1. Types of Boundary Conditions
          2. Testing the Boundary Edges
        2. Sub-Boundary Conditions
          1. Powers-of-Two
          2. ASCII Table
        3. Default, Empty, Blank, Null, Zero, and None
        4. Invalid, Wrong, Incorrect, and Garbage Data
      5. State Testing
        1. Testing the Software's Logic Flow
          1. Creating a State Transition Map
          2. Reducing the Number of States and Transitions to Test
          3. What to Specifically Test
        2. Testing States to Fail
          1. Race Conditions and Bad Timing
          2. Repetition, Stress, and Load
      6. Other Black-Box Test Techniques
        1. Behave Like a Dumb User
        2. Look for Bugs Where You've Already Found Them
        3. Think like a Hacker
        4. Follow Experience, Intuition, and Hunches
      7. Summary
      8. Quiz
    3. 6. Examining the Code
      1. Static White-Box Testing: Examining the Design and Code
      2. Formal Reviews
        1. Peer Reviews
        2. Walkthroughs
        3. Inspections
      3. Coding Standards and Guidelines
        1. Examples of Programming Standards and Guidelines
        2. Obtaining Standards
      4. Generic Code Review Checklist
        1. Data Reference Errors
        2. Data Declaration Errors
        3. Computation Errors
        4. Comparison Errors
        5. Control Flow Errors
        6. Subroutine Parameter Errors
        7. Input/Output Errors
        8. Other Checks
      5. Summary
      6. Quiz
    4. 7. Testing the Software with X-Ray Glasses
      1. Dynamic White-Box Testing
      2. Dynamic White-Box Testing Versus Debugging
      3. Testing the Pieces
        1. Unit and Integration Testing
        2. An Example of Module Testing
      4. Data Coverage
        1. Data Flow
        2. Sub-Boundaries
        3. Formulas and Equations
        4. Error Forcing
      5. Code Coverage
        1. Program Statement and Line Coverage
        2. Branch Coverage
        3. Condition Coverage
      6. Summary
      7. Quiz
  8. III. Applying Your Testing Skills
    1. 8. Configuration Testing
      1. An Overview of Configuration Testing
        1. Isolating Configuration Bugs
        2. Sizing Up the Job
      2. Approaching the Task
        1. Decide the Types of Hardware You'll Need
        2. Decide What Hardware Brands, Models, and Device Drivers Are Available
        3. Decide Which Hardware Features, Modes, and Options Are Possible
        4. Pare Down the Identified Hardware Configurations to a Manageable Set
        5. Identify Your Software's Unique Features That Work with the Hardware Configurations
        6. Design the Test Cases to Run on Each Configuration
        7. Execute the Tests on Each Configuration
        8. Rerun the Tests Until the Results Satisfy Your Team
      3. Obtaining the Hardware
      4. Identifying Hardware Standards
      5. Configuration Testing Other Hardware
      6. Summary
      7. Quiz
    2. 9. Compatibility Testing
      1. Compatibility Testing Overview
      2. Platform and Application Versions
        1. Backward and Forward Compatibility
        2. The Impact of Testing Multiple Versions
      3. Standards and Guidelines
        1. High-Level Standards and Guidelines
        2. Low-Level Standards and Guidelines
      4. Data Sharing Compatibility
      5. Summary
      6. Quiz
    3. 10. Foreign-Language Testing
      1. Making the Words and Pictures Make Sense
      2. Translation Issues
        1. Text Expansion
        2. ASCII, DBCS, and Unicode
        3. Hot Keys and Shortcuts
        4. Extended Characters
        5. Computations on Characters
        6. Reading Left to Right and Right to Left
        7. Text in Graphics
        8. Keep the Text out of the Code
      3. Localization Issues
        1. Content
        2. Data Formats
      4. Configuration and Compatibility Issues
        1. Foreign Platform Configurations
        2. Data Compatibility
      5. How Much Should You Test?
      6. Summary
      7. Quiz
    4. 11. Usability Testing
      1. User Interface Testing
      2. What Makes a Good UI?
        1. Follows Standards and Guidelines
        2. Intuitive
        3. Consistent
        4. Flexible
        5. Comfortable
        6. Correct
        7. Useful
      3. Testing for the Disabled: Accessibility Testing
        1. Legal Requirements
        2. Accessibility Features in Software
      4. Summary
      5. Quiz
    5. 12. Testing the Documentation
      1. Types of Software Documentation
      2. The Importance of Documentation Testing
      3. What to Look for When Reviewing Documentation
      4. The Realities of Documentation Testing
      5. Summary
      6. Quiz
    6. 13. Testing for Software Security
      1. WarGames—the Movie
      2. Understanding the Motivation
      3. Threat Modeling
      4. Is Software Security a Feature? Is Security Vulnerability a Bug?
      5. Understanding the Buffer Overrun
      6. Using Safe String Functions
      7. Computer Forensics
      8. Summary
      9. Quiz
    7. 14. Website Testing
      1. Web Page Fundamentals
      2. Black-Box Testing
        1. Text
        2. Hyperlinks
        3. Graphics
        4. Forms
        5. Objects and Other Simple Miscellaneous Functionality
      3. Gray-Box Testing
      4. White-Box Testing
      5. Configuration and Compatibility Testing
      6. Usability Testing
      7. Introducing Automation
      8. Summary
      9. Quiz
  9. IV. Supplementing Your Testing
    1. 15. Automated Testing and Test Tools
      1. The Benefits of Automation and Tools
      2. Test Tools
        1. Viewers and Monitors
        2. Drivers
        3. Stubs
        4. Stress and Load Tools
        5. Interference Injectors and Noise Generators
        6. Analysis Tools
      3. Software Test Automation
        1. Macro Recording and Playback
        2. Programmed Macros
        3. Fully Programmable Automated Testing Tools
      4. Random Testing: Monkeys and Gorillas
        1. Dumb Monkeys
        2. Semi-Smart Monkeys
        3. Smart Monkeys
      5. Realities of Using Test Tools and Automation
      6. Summary
      7. Quiz
    2. 16. Bug Bashes and Beta Testing
      1. Having Other People Test Your Software
      2. Test Sharing
      3. Beta Testing
      4. Outsourcing Your Testing
      5. Summary
      6. Quiz
  10. V. Working with Test Documentation
    1. 17. Planning Your Test Effort
      1. The Goal of Test Planning
      2. Test Planning Topics
        1. High-Level Expectations
        2. People, Places, and Things
        3. Definitions
        4. Inter-Group Responsibilities
        5. What Will and Won't Be Tested
        6. Test Phases
        7. Test Strategy
        8. Resource Requirements
        9. Tester Assignments
        10. Test Schedule
        11. Test Cases
        12. Bug Reporting
        13. Metrics and Statistics
        14. Risks and Issues
      3. Summary
      4. Quiz
    2. 18. Writing and Tracking Test Cases
      1. The Goals of Test Case Planning
      2. Test Case Planning Overview
        1. Test Design
        2. Test Cases
        3. Test Procedures
          1. Detail Versus Reality
      3. Test Case Organization and Tracking
      4. Summary
      5. Quiz
    3. 19. Reporting What You Find
      1. Getting Your Bugs Fixed
      2. Isolating and Reproducing Bugs
      3. Not All Bugs Are Created Equal
      4. A Bug's Life Cycle
      5. Bug-Tracking Systems
        1. The Standard: The Test Incident Report
        2. Manual Bug Reporting and Tracking
        3. Automated Bug Reporting and Tracking
      6. Summary
      7. Quiz
    4. 20. Measuring Your Success
      1. Using the Information in the Bug Tracking Database
      2. Metrics That You'll Use in Your Daily Testing
      3. Common Project-Level Metrics
      4. Summary
      5. Quiz
  11. VI. The Future
    1. 21. Software Quality Assurance
      1. Quality Is Free
      2. Testing and Quality Assurance in the Workplace
        1. Software Testing
        2. Quality Assurance
        3. Other Names for Software Testing Groups
      3. Test Management and Organizational Structures
      4. Capability Maturity Model (CMM)
      5. ISO 9000
      6. Summary
      7. Quiz
    2. 22. Your Career as a Software Tester
      1. Your Job as a Software Tester
      2. Finding a Software Testing Position
      3. Gaining Hands-On Experience
      4. Formal Training Opportunities
      5. Websites
      6. Professional Organizations Dedicated to Software or Software Quality
      7. Further Reading
      8. Summary
      9. Quiz
    3. A. Answers to Quiz Questions
      1. Chapter 1
      2. Chapter 2
      3. Chapter 3
      4. Chapter 4
      5. Chapter 5
      6. Chapter 6
      7. Chapter 7
      8. Chapter 8
      9. Chapter 9
      10. Chapter 10
      11. Chapter 11
      12. Chapter 12
      13. Chapter 13
      14. Chapter 14
      15. Chapter 15
      16. Chapter 16
      17. Chapter 17
      18. Chapter 18
      19. Chapter 19
      20. Chapter 20
      21. Chapter 21
      22. Chapter 22