Software Test Attacks to Break Mobile and Embedded Devices

Software Test Attacks to Break Mobile and Embedded Devices

by Jon Duncan Hagar
Released September 2013
Publisher(s): Chapman and Hall/CRC
ISBN: 9781498760140

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

This book presents an attack basis for testing mobile and embedded systems in "smart" devices. It explains patterns and techniques ranging from simple mind mapping to sophisticated test labs. For traditional testers moving into the mobile and embedded area, the book bridges the gap between IT and mobile/embedded system testing. It illustrates how to apply both traditional and new approaches. For those working with mobile/embedded systems without an extensive background in testing, the book brings together testing ideas, techniques, and solutions that are immediately applicable to testing smart and mobile devices.

Table of contents

  1. Foreword by Dorothy Graham
  2. Foreword by Lisa Crispin
  3. Preface
  4. Acknowledgments
  5. Copyright and Trademarks Declaration Page
  6. Introduction
    1. Getting Started
      1. Who Should Be Familiar with This Book
      2. What Is Not in This Book
      3. Relationship to Other Test References and Works by Other Authors
      4. Attack-Based Testing and Different Approaches to Software Development
      5. A Final Introduction Warning
    2. References
  7. Author
  8. Chapter 1 - Setting the Mobile and Embedded Framework
    1. Objectives of Testing Mobile and Embedded Software Systems
    2. What Is Embedded Software?
    3. What Are “Smart” Handheld and Mobile Systems?
    4. Why Mobile and Embedded Attacks?
    5. Framework for Attacks
    6. Beginning Your Test Strategy
    7. Attacks on Mobile and Embedded Software
    8. If You Are New to Testing
    9. An Enlightened Tester Makes a Better Tester
    10. Exercises (Answers Are on My Website)
    11. References
  9. Chapter 2 - Developer Attacks: Taking the Code Head On
    1. Attack 1: Static Code Analysis
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    2. Attack 2: Finding White-Box Data Computation Bugs
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Attack Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Determine If This Attack Exposes Failures
      6. How to Conduct This Attack
    3. Attack 3: White-Box Structural Logic Flow Coverage
      1. Who Conducts This Attack and When to Apply This Attack?
      2. What Faults Make This Attack Successful?
      3. How to Determine If This Attack Exposes Failures
      4. How to Conduct This Attack
    4. Test Coverage Concepts for White-Box Structural Testing
    5. Note of Concern in Mobile and Embedded Environments
    6. Exercises (Answers on My Website)
    7. References
  10. Chapter 3 - Control System Attacks
    1. Attack 4: Finding Hardware–System Unhandled Uses in Software
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    2. Attack 5: Hardware-to-Software and Software-to-Hardware Signal Interface Bugs
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    3. Attack 6: Long-Duration Control Attack Runs
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where to Conduct This Attack?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    4. Attack 7: Breaking Software Logic and/or Control Laws
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Performs This Attack?
      4. Where to Run This Attack?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    5. Attack 8: Forcing the Unusual Bug Cases
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where to Conduct This Attack?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    6. Exercises (Answers Are on My Website)
    7. References
  11. Chapter 4 - Hardware Software Attacks
    1. Attack 9: Breaking Software with Hardware and System Operations
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where to Conduct This Attack?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    2. Sub-Attack 9.1: Breaking Battery Power
    3. Attack 10: Finding Bugs in Hardware–Software Communications
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    4. Attack 11: Breaking Software Error Recovery
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    5. Attack 12: Interface and Integration Testing
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Attack Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Conduct This Attack
    6. Sub-Attack 12.1: Configuration Integration Evaluation
    7. Attack 13: Finding Problems in Software–System Fault Tolerance
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    8. Exercises (Answers Are on My Website)
    9. References
  12. Chapter 5 - Mobile and Embedded Software Attacks
    1. Attack 14: Breaking Digital Software Communications
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes a Failure?
      6. How to Conduct This Attack
    2. Attack 15: Finding Bugs in the Data
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    3. Attack 16: Bugs in System-Software Computation
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where to Conduct This Attack?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    4. Attack 17: Using Simulation and Stimulation to Drive Software Attacks
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Attack Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    5. Exercises (Answers Are on My Website)
    6. References
  13. Chapter 6 - Time Attacks: “It’s about Time”
    1. Attack 18: Bugs in Timing Interrupts and Priority Inversions
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    2. State Modeling Example
    3. Attack 19: Finding Time-Related Bugs
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    4. Attack 20: Time-Related Scenarios, Stories, and Tours
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where to Conduct This Attack?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    5. Attack 21: Performance Testing Introduction
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is the Test Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Determine If This Attack Exposes Failures
      6. How to Conduct This Attack
    6. Supporting Concepts
    7. Completing and Reporting the Performance Attack
    8. Wrapping Up
    9. Exercises (Answers Are on My Website)
    10. References
  14. Chapter 7 - Human User Interface Attacks: “The Limited (and Unlimited) User Interface”
    1. How to Get Started—the UI
    2. Attack 22: Finding Supporting (User) Documentation Problems
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    3. Sub-Attack 22.1: Confirming Install-Ability
    4. Attack 23: Finding Missing or Wrong Alarms
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    5. Attack 24: Finding Bugs in Help Files
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where to Conduct This Attack?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    6. Exercises (Answers Are on My Website)
    7. References
  15. Chapter 8 - Smart and/or Mobile Phone Attacks
    1. General Notes and Attack Concepts Applicable to Most Mobile–Embedded Devices
    2. Attack 25: Finding Bugs in Apps
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If This Attack Exposes Failures
      6. How to Conduct This Attack
    3. Attack 26: Testing Mobile and Embedded Games
      1. When to Apply This Attack
      2. What Faults Make This Attack Successful?
      3. Who Conducts This Attack?
      4. Where Is This Attack Conducted?
      5. How to Determine If This Attack Exposes Failures
      6. How to Conduct This Attack
    4. Attack 27: Attacking App–Cloud Dependencies
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Attack Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Determine If the Attack Exposes Failures
      6. How to Conduct This Attack
    5. Exercises (Answers Are on My Website)
    6. References
  16. Chapter 9 - Mobile/Embedded Security
    1. The Current Situation
    2. Reusing Security Attacks
    3. Attack 28: Penetration Attack Test
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Attack Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Determine If This Attack Exposes Failures
      6. How to Conduct This Attack
    4. Attack 28.1: Penetration Sub-Attacks: Authentication—Password Attack
      1. How to Conduct This Attack
    5. Attack 28.2: Sub-Attack Fuzz Test
      1. How to Conduct This Attack
    6. Attack 29: Information Theft—Stealing Device Data
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where to Conduct This Attack?
      4. How to Determine If This Attack Exposes Failures
      5. What Faults Make This Attack Successful?
      6. How to Conduct This Attack
    7. Attack 29.1: Sub-Attack—Identity Social Engineering
    8. Attack 30: Spoofing Attacks
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Attack Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Determine If This Attack Exposes Failures
      6. How to Conduct This Attack
    9. Attack 30.1: Location and/or User Profile Spoof Sub-Attack
    10. Attack 30.2: GPS Spoof Sub-Attack
    11. Attack 31: Attacking Viruses on the Run in Factories or PLCs
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Attack Conducted?
      4. How to Determine If the Attack Exposes Failures
      5. How to Conduct This Attack
    12. Exercises (Answers Are on My Website)
    13. References
  17. Chapter 10 - Generic Attacks
    1. Attack 32: Using Combinatorial Tests
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Attack Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Determine If This Attack Exposes Failures
      6. How to Conduct This Attack
    2. Attack 33: Attacking Functional Bugs
      1. When to Apply This Attack
      2. Who Conducts This Attack?
      3. Where Is This Test Conducted?
      4. What Faults Make This Attack Successful?
      5. How to Determine If This Attack Exposes Failures
      6. How to Conduct This Attack
    3. Exercises (Answers Are on My Website)
    4. References
  18. Chapter 11 - Mobile and Embedded System Labs
    1. Introduction to Labs
    2. To Start
    3. Test Facilities
    4. Why Should a Tester Care?
    5. What Problem Does a Test Lab Solve?
    6. Staged Evolution of a Test Lab
    7. Simulation Environments
    8. Prototype and Early Development Labs
    9. Development Support Test Labs
    10. Integration Labs
    11. Pre-Product and Product Release (Full Test Lab)
    12. Field Labs
    13. Other Places Labs Can Be Realized
    14. Developing Labs: A Project inside of a Project
    15. Planning Labs
    16. Requirement Considerations for Labs
    17. Functional Elements for a Developer Support Lab
    18. Functional Elements for a Software Test Lab
    19. Test Lab Design Factors
    20. Lab Implementation
    21. Lab Certification
    22. Operations and Maintenance in the Lab
    23. Lab Lessons Learned
    24. Automation Concepts for Test Labs
    25. Tooling to Support Lab Work
    26. Test Data Set-Up
    27. Test Execution: For Developer Testing
    28. Test Execution: General
    29. Product and Security Analysis Tools
    30. Tools for the Lab Test Results Recording
    31. Performance Attack Tooling
    32. Basic and Generic Test Support Tools
    33. Automation: Test Oracles for the Lab Using Modeling Tools
    34. Simulation, Stimulation, and Modeling in the Lab Test Bed
    35. Continuous Real-Time, Closed-Loop Simulations to Support Lab Test Environments
    36. Keyword-Driven Test Models and Environments
    37. Data Collection, Analysis, and Reporting
    38. PostTest Data Analysis
    39. PostTest Data Reporting
    40. Wrap Up: N-Version Testing Problems in Labs and Modeling
    41. Final Thoughts: Independence, Blind Spots, and Test Lab Staffing
    42. Exercises (Answers Are on My Website)
    43. References
  19. Chapter 12 - Some Parting Advice
    1. Are We There Yet?
    2. Will You Get Started Today?
    3. Advice for the “Never Ever” Tester
    4. Bug Database, Taxonomies, and Learning from Your History
    5. Lessons Learned and Retrospectives
    6. Implementing Software Attack Planning
    7. Regression and Retest
    8. Where Do You Go from Here?
    9. Exercises (Answers Are on My Website)
    10. References
  20. Appendix A: Mobile and Embedded Error Taxonomy: A Software Error Taxonomy (for Testers)
    1. Introduction
    2. Characteristics of the Enemy: A Taxonomy
    3. Summary of the Taxonomy Table
    4. Some Details and Implications
    5. Where Did the Data Come From?
    6. What Does the Taxonomy Indicate?
    7. Changing the Face of Mobile and Embedded Software
    8. Summary
    9. References
  21. Appendix B: Mobile and Embedded Coding Rules
    1. Coding Guidance (to Be Followed) and the Evil of Code (to Be Avoided)
    2. Website
  22. Appendix C: Quality First: “Defending the Source Code So That Attacks Are Not So Easy”
    1. Introduction
      1. Why Would Anyone Want to Make Testing Harder?
        1. Who Should Do This?
        2. When to Apply This Attack
        3. How to Conduct This Attack
    2. Standards and Commonality: The Dark Side of Commercial or Third-Party Software, Code Reuse, and Operating Systems
    3. Off-the-Shelf and Software Reuse
    4. Everyone Hates Doing Documentation but Surely Misses It When They Need It
    5. Is Testing Dead?
    6. Summary: The P Word: Processes and Standards
    7. Bibliography
  23. Appendix D: Basic Timing Concepts
    1. Basic Timing Concepts and Terms
    2. Impacts of Timing
  24. Appendix E: Detailed Mapping of Attacks
  25. Appendix F: UI/GUI and Game Evaluation Checklist
    1. Checklist of Usability Factors and Criteria
    2. References
  26. Appendix G: Risk Analysis, FMEA, and Brainstorming
    1. FMEA/FMECA
    2. Brainstorming (Organized)
    3. References
  27. References
    1. References and Additional Reading
      1. Books on My Bookshelf and Many Used as References
      2. Sampling of Standards That Testers Should Be Familiar With (Used in the Production of This Book)
      3. Tool Information (To Name Only a Few)
    2. Mobile Embedded News and Websites
      1. Industry Communication Standards and Standard Location Websites (To Name Only a Few)
  28. Glossary
    1. Term Definition and/or Reference

Product information

  • Title: Software Test Attacks to Break Mobile and Embedded Devices
  • Author(s): Jon Duncan Hagar
  • Release date: September 2013
  • Publisher(s): Chapman and Hall/CRC
  • ISBN: 9781498760140