book

Critical Testing Processes: Plan, Prepare, Perform, Perfect

by Rex Black

July 2003

Intermediate to advanced

608 pages

16h 15m

English

Addison-Wesley Professional

Read now

Unlock full access

Dedication
Book ScopeWho Should Read This Book?Demystifying ProcessesTypes and Relationships of Critical Testing ProcessesCritical Testing Processes in ContextSumatra, a Hypothetical ProjectTerminologyOn Using This BookRecognize Good ProcessesHandle ChallengesImplement ImprovementsTo Your Testing Success!
A Testing ProcessUnderstanding Operational and Organizational ContextIntroducing the Sumatra Project and the SpeedyWriter Test TeamTesting within the System Development LifecycleOrganizing the Testers and the TestingManagement DimensionsBeyond Testing Process Context
A Quality Risk Analysis ProcessJamal Introduces the Team to Quality Risk AnalysisRecognize a Good Quality Risk Analysis ProcessInvolve the Right ParticipantsEmploy the Right TechniqueSeparate the Vital Few from the Trivial ManyFind Potential Bugs before TestingStart at the Right TimeAdapt to ChangeHandle ChallengesGetting People to Get Real about RiskQuality Risks from Various SourcesWho Drives the Quality Risk Analysis Bus?Eliminating the Appearance of Magic in the Quantification of RiskDealing with Variation in RatingsConvincing Stakeholders to Invest Time and Accept RealityImplement Improvements
An Estimation ProcessJamal Predicts the FutureUsing Rules of Thumb for Rough EstimationEffort, Duration, and DependenciesPlanningStaffingTest DevelopmentTest Environment ConfigurationTest ExecutionMoving on to Dollars and Cents

Analyzing Return on Investment for TestingBugs That Get Fixed—or PreventedBugs That Don't Get Fixed—but Are KnownTests That Mitigate RisksGuiding the ProjectA Final Note on Return on Investment EstimatesJamal Prepares the Bill—and the JustificationMoving Forward, Then Looking Back
Selling the EstimateJamal Makes His CaseRecognize a Good Estimation ProcessUnderstand the Factors that Affect the EstimatePaint a Realistic Picture of the Project's FutureAssign Responsibility for ActionPredict with HonestyEstimate QuicklyHandle ChallengesWhat Kind of Investment Is Testing, Really?It Might Look Like Magic—but It's Only an EstimateThe Nine-Month RuleBe Careful with NonstakeholdersWhen Selling FailsSafety- or Mission-Critical SystemsImplement Improvements
A Test Planning ProcessJamal Has a PlanBeyond the Process: Key Considerations for the Test PlanSelecting Strategies for TestingA Special Case: Regression Risk Management StrategiesUnderstanding Test Environment Execution ResourcesManaging Test Systems and Environment ConfigurationsTest Subproject RisksGetting Everyone On Board
Jamal Makes His PitchRecognize a Good Test Planning ProcessEstablish Clear Criteria for PhasesEstablish Consensus, Common Expectations, and CommitmentComplete at a Reasonable TimePromote Reasonable Flexibility and CreativityProduce Appropriate DocumentationProvide Opportunities to Catch ErrorsExploit Synergies with the Project, Development, Build, and Integration PlansHandle ChallengesTranscending the TemplatePlanning for OutsourcingWhen Key Players Don't Support the Test PlanObeying the Law and Pleasing RegulatorsImplement Improvements
A Team Building ProcessVariations on the Team Building ProcessThe Sumatra Test Team GrowsIdentifying and Managing the Critical Test Team SkillsRegarding Test TechniciansTest Team Staffing Variations: Temporary Assignment, Rotation, Farm Team, and BackwaterEducation, Training, Certification, and the Professionalization of TestingAttitude Matters TooBeyond the Obvious Interview QuestionsFrom Adding People to Growing Skills
Jamal and Lin-Tsu Discuss Skills GrowthUsing Skills Assessment as a Career Growth and Team Management ToolRecognize a Good Team Building ProcessOperate on a Win-Win PhilosophyAccurately Define the JobProvide the Employee with a Long-Term Career PathGive Employees Appropriate and Fair SalariesRetain Employees with Sufficient Education, Experience, and SkillAllow Appropriate SpecializationHandle ChallengesWhat Do Testers Need to Know—and What Can You Get?White ElephantsUsing Contractors and ConsultantsImplement Improvements
A Test System Design and Implementation ProcessEmma Creates Stress on the JobThree Crucial ConsiderationsTest OraclesAppropriate DocumentationCombinatorial ExplosionsShifting to the Management Picture
A Brief Survey of Test Coverage Analysis TechniquesJamal Assesses CoverageRecognize a Good Test System Design and Implementation ProcessProvides Capable Tools for Assessment of QualityEfficiently Implements, Uses, Reuses, and Maintains the Test SystemSelects the Right TechniquesPrevents BugsHandle ChallengesHow Many Conditions in a Test Case?Retesting Third-Party ComponentsVaguely Defined Requirements, Usage Profiles, and EnvironmentsBalancing Coverage, Schedule, and BudgetImplement Improvements
A Test Release ProcessThe Big Build Hits the Test LabRecognize a Good Test Release ProcessProvide Installable, Improved, and Stable Test ReleasesProduce Predictable, Timely, and Regular Test ReleasesUse Defined, Customer-Like Install and Uninstall ProcessesGive the Test Release a Name That It KnowsBuild from Carefully Selected, Coordinated, and Documented ContentHandle ChallengesWho Owns the Test Release Processes?Complexity of the System Under Test—and BeyondIncomplete and Unrealistic System and Acceptance Test ReleasesImplement Improvements
A Test Execution ProcessTest Tracking WorksheetsThe Attack on the Big BuildRecognize a Good Test Execution ProcessFind the Scary Stuff FirstProduce, Gather, and Disseminate Valuable InformationCorrectly Interpret Test ResultsExhibit the Proper AttitudesProceed EfficientlyHandle ChallengesDealing with Extended Shifts and Outsourced TestingAccommodating Holidays, Cultures, and VacationsCapturing the History on the GroundImplement Improvements
A Bug Reporting ProcessA Big Bug in the Big BuildBeyond the Failure DescriptionRecognize a Good Bug Reporting ProcessEffectively Communicate Useful InformationDescribe One Symptom per Report and File One Report per SymptomDraw a Clear Boundary between Testing and DebuggingHandle ChallengesSeparating Bugs, Features, and “So What”Handling Bugs That Got Fixed by Accident and Irreproducible SymptomsKeeping Noise out of the SignalBuilding Trust and Credibility with ProgrammersSelecting the Right Bug Tracking ToolImplement Improvements
A Test Results Reporting ProcessJamal Defines a Dashboard—and Reports on the Big Build's Test ResultsRecognize a Good Test Results Reporting ProcessDeliver Useful, Credible, and Timely InformationRelate Test Status to Project Progress—and Influence Future ProgressShow Trends and Destinations in Quality over TimeUse Appropriate Mechanisms and DocumentsTune the Communication to Each ListenerHandle ChallengesHandling the PresentationThe Bearer of Bad NewsAn Ever-Changing Complex StoryMaintaining a Passion for Quality While Giving a Dispassionate PresentationImplement Improvements
A Change Management ProcessJamal Makes a Case—and AccommodatesAn Interconnected ProcessRecognize a Good Change Management ProcessSelect the Right Changes in the Right OrderBalance Considerations of Features, Schedule, Budget, and QualityHandle ChallengesThe Complex Nature of Change EffectsThe Ripple Effects of ChangeBecoming a Roadblock—or Becoming Perceived as a RoadblockImplement Improvements
Recognize a Good Test ProcessProvide Valuable, Economical ServicesPervade the ProjectUse a Mature ProcessApply the Appropriate Level of Formality and StandardizationEmploy Appropriate StrategiesSuccessfully Introduce ToolsEnjoy Continuous Learning and InspirationHandle ChallengesHow Can We Measure What We Can't Define?Immature Development or Maintenance ProcessesUnrealistic ExpectationsRelationships“Addicted to Test”Incremental Process ImprovementFine-Tune the Big ProcessesIdentify and Remove the Big Process ObstaclesImplement ImprovementsJamal Brown Looks Back—and Looks ForwardConclusion

Content preview from Critical Testing Processes: Plan, Prepare, Perform, Perfect

Chapter 13. Assess Quality: Execute Test Cases

Finally the time has come to do some testing. During the test execution process, the testers will use whatever test system we have put together to assess the quality of the test releases as we receive them. This is both a potentially exciting and a potentially scary point in the project because the quality of the system is unknown until the testers start pounding on it. Furthermore, the test team generally has a unique charter here, that of looking for ways to make the system under test get as far out of control and away from the expected as possible.

In addition, while all managers are expected to manage unexpected events within their areas of responsibility, the test manager is the only manager in ...