book

Beautiful Testing

Name: Beautiful Testing
ISBN: 9781449388683

by Adam Goucher, Tim Riley

October 2009

Beginner

350 pages

11h 10m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Beautiful Testing
SPECIAL OFFER: Upgrade this ebook with O’Reilly
A Note Regarding Supplemental Files
Preface
How This Book Is OrganizedPart I, Beautiful TestersPart II, Beautiful ProcessPart III, Beautiful Tools
Using Code Examples
Safari® Books Online
How to Contact Us
Acknowledgments
I. Beautiful Testers
1. Was It Good for You?

2. Beautiful Testing Satisfies Stakeholders
For Whom Do We Test?
What Satisfies?
What Beauty Is External?
What Beauty Is Internal?
Conclusions
3. Building Open Source QA Communities
Communication
Volunteers
Coordination
You Are Not SpecialYou Are Always RecruitingYou Are Always Engaging
Events
PublicityWhat We Actually DidGoal Setting and Rewards
Conclusions
4. Collaboration Is the Cornerstone of Beautiful Performance Testing
Setting the Stage
100%?!? Fail
OK, but What’s a Performance Test Case?You Can’t Performance Test Everything
The Memory Leak That Wasn’t
Can’t Handle the Load? Change the UI
It Can’t Be the Network
It’s Too Slow; We Hate It
Wrap-Up
II. Beautiful Process
5. Just Peachy: Making Office Software More Reliable with Fuzz Testing
User Expectations
What Is Fuzzing?
Why Fuzz Test?
Improve InteroperabilityImprove User SatisfactionImprove Security
Fuzz Testing
PreparationGeneral FuzzingCustom FuzzingRandom FuzzingLimitations
Future Considerations
6. Bug Management and Test Case Effectiveness
Bug ManagementThe First Bug FoundIt Isn’t “Just a Bug”
The First Step in Managing a Defect Is Defining It
Who?What?When?Where?To a developer, it is clear that a defect lives in the source codeTo an end user/support person, defects live in the system or distributionOne approach to multi-code-base defect trackingTagsTagged Defect Sets (Tag Clouds): Why?
Test Case Effectiveness
Capturing the Impact of Bug SeverityAnalyzing Test Escape Bugs
Case Study of the OpenSolaris Desktop Team
AssumptionsThe Process of Calculating the TCE MetricCorrective action
Conclusions
Acknowledgments
References
7. Beautiful XMPP Testing
Introduction
XMPP 101
Testing XMPP Protocols
Unit Testing Simple Request-Response Protocols
Unit Testing Multistage Protocols
Testing Session Initialization
Automated Interoperability Testing
Diamond in the Rough: Testing XML Validity
Conclusions
References
8. Beautiful Large-Scale Test Automation
Before We Start
What Is Large-Scale Test Automation?
The Basics of a Test Automation SystemA Beautiful System
The First Steps
Test Infrastructure Is CriticalTest Collateral
Automated Tests and Test Case Management
The Automated Test Lab
Deploying the Test BedOther Considerations
Test Distribution
Failure Analysis
Reporting
Putting It All Together
9. Beautiful Is Better Than Ugly
The Value of Stability
Ensuring Correctness
The Buildbot SystemRefleak TestingDoc TestingRelease TestingDynamic AnalysisStatic Analysis
Conclusions
10. Testing a Random Number Generator
What Makes Random Number Generators Subtle to Test?
Uniform Random Number Generators
Nonuniform Random Number Generators
A Progression of Tests
Range TestsMean TestVariance TestBucket TestKolmogorov-Smirnov Test
Conclusions
11. Change-Centric Testing
How to Set Up the Document-Driven, Change-Centric Testing Framework?
Change-Centric Testing for Complex Code Development Models
Steps to Change-Centric TestingUnderstanding the caller–callee dependency and test case to source file mappingGenerating the call graph of caller–callee functions in the executableExample 1Understanding the change impactUnderstanding the code coverage and gap analysisExample 2Example 3
What Have We Learned So Far?
Example 4Scenario 1Scenario 2Scenario 3
Conclusions
12. Software in Use
A Connection to My Work
From the Inside
Adding Different Perspectives
Exploratory, Ad-Hoc, and Scripted Testing
Multiuser Testing
The Science Lab
Simulating Real Use
Testing in the Regulated World
At the End
13. Software Development Is a Creative Process
Agile Development As Performance
Practice, Rehearse, Perform
Evaluating the Ineffable
Two Critical Tools
Software Testing Movements
The Beauty of Agile Testing
QA Is Not Evil
Beauty Is the Nature of This Work
References
14. Test-Driven Development: Driving New Standards of Beauty
Beauty As Proportion and Balance
Agile: A New Proportion and Balance
Test-Driven Development
Examples Versus Tests
Readable Examples
Permanent Requirement Artifacts
Testable Designs
Tool Support
AuthorReaderExecutorsResult consumerReport consumer
Team Collaboration
Experience the Beauty of TDD
References
15. Beautiful Testing As the Cornerstone of Business Success
The Whole-Team ApproachNo Story Is “Done” Until It’s TestedContinuous ImprovementDelivering Maximum Business ValueFocusing on Testability
Automating Tests
Choosing Tools by Team ConsensusIncremental AutomationFlipping the Pyramid
Driving Development with Tests
ExperimentingPlanningDriving Coding with ExamplesBumps in the Road
Delivering Value
Everyone Owns Business ProblemsIncremental Testing and CodingPlan Just Enough Ahead
A Success Story
Post Script
16. Peeling the Glass Onion at Socialtext
It’s Not Business…It’s Personal
Tester Remains On-Stage; Enter Beauty, Stage Right
Come Walk with Me, The Best Is Yet to Be
Automated Testing Isn’t
Into Socialtext
But…What Do You Make?Software Process at SocialtextWhat We Actually DoWikitests
A Balanced Breakfast Approach
Regression and Process Improvement
The Last Pieces of the Puzzle
Acknowledgments
17. Beautiful Testing Is Efficient Testing
SLIMES: SecurityL: LanguagesI: requIrementsM: MeasurementE: Existing
Scripting
Discovering Developer Notes
LOUD
Oracles and Test Data Generation
Mindmaps
Efficiency Achieved
III. Beautiful Tools
18. Seeding Bugs to Find Bugs: Beautiful Mutation Testing
Assessing Test Suite Quality
Watching the Watchmen
Efficient Mutation TestingSelecting Mutation Operators
An AspectJ Example
Equivalent Mutants
Focusing on Impact
The Javalanche Framework
Odds and Ends
Acknowledgments
References
19. Reference Testing As Beautiful Testing
Reference Test Structure
Reference Test Extensibility
Asynchronous TestsPrinting TestsInvalidation Tests
Building Community
20. Clam Anti-Virus: Testing Open Source with Open Tools
The Clam Anti-Virus Project
Testing Methods
Black Box Versus White Box TestingStatic AnalysisGCCClang Static AnalyzerSplintBe patientMemory CheckersValgrindElectric Fence and DUMAMudflapLimitationsUnit TestingTest ScriptsFuzz TestingCollection of Problematic FilesTesting the EnvironmentBuildbotCompatibility TestingPerformance TestingTesting for False Positive and False Negative AlertsFalse positivesFalse negativesUsability and User Acceptance TestingOpinion pollingRelease candidates
Summary
Credits
21. Web Application Testing with Windmill
Introduction
Overview
Writing Tests
Running TestsDebugging TestFirebug LiteLookupNode
The Project
Comparison
Conclusions
References
22. Testing One Million Web Pages
In the Beginning…
The Tools Merge and Evolve
The Nitty-Gritty
Summary
Acknowledgments
23. Testing Network Services in Multimachine Scenarios
The Need for an Advanced Testing Tool in eBox
Development of ANSTE to Improve the eBox QA Process
How eBox Uses ANSTE
Sample ANSTE Tests for eBox
How Other Projects Can Benefit from ANSTE
A. Contributors
Index
Colophon
SPECIAL OFFER: Upgrade this ebook with O’Reilly

404

Not Found

Either this content does not exist or you do not have access to it. Try searching for something new.

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Beautiful Testing

by Adam Goucher, Tim Riley

404

Not Found