book

Software Testing Strategies

by Matthew Heusser, Michael Larsen

December 2023

Intermediate to advanced

378 pages

11h 54m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorsAbout the reviewers:
Our scope - beyond button-pushingWhat is Testing?Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touchShare your thoughtsDownload a free PDF copy of this book
Jumping into testingThe impossibility of complete testingWhat is the EVP asking for?Toward a theory of errorTesting software – an exampleTesting software – an analysisQuick attacks – the airdropped testerTest design – input space coverageEquivalence classes and boundariesDecision tablesDecision treesAll-pairs and pairwise testingHigh volume automated approachesOther approachesData and predictability – the oracle problemSummaryFurther reading
Technical requirementsNo silver bullets – you can’t radically fix the test cycleThe minefield regression problemCoverage modelThe Battleships problem – testing versus checkingComparing Battleships to bugsAutomation rarely beats human intuitionThe maintenance problemThe dangers of comprehensive testsThe money problemLessons from the fundamental issuesSummary
Technical requirementsThe programmer’s viewTesting and reliabilityThe hexagonal architectureIntroducing FizzBuzzUnit testsTDDConsequences of TDDUnit test and unit code designUsing test doubles to create seamsMutation testingWeb APIs from a test perspectiveWeb API testing strategyTesting functional and legacy codeA Roman Numerals KataSummary
Technical requirementsA word of warningHuman Or Tooling—is it either/or?GUI test automation patternsEliminating redundancy with domain-specific librariesEliminating redundancy through object locatorsDo you need conditionals, looping structures, and variables?The tradeoff between assertion and image recognitionDesigning your own systemToward specification by exampleSpecification by exampleLow-code and no-code test automationBatch- and model-driven test automationThis chapter is completely wrongSummary
Technical requirementsUnderstanding load and performance testingGetting to know the basics of load testingSetting up a load testExploring security testingSecurity conceptsChecking out common security vulnerabilitiesLearning about industry standardsInvestigating some security testing toolsDelving into accessibility testingWhat is accessibility?Advocating for accessibilityInvestigating the distinctions between accessibility and inclusive designLearning about the WCAG standardInvestigating some accessibility testing toolsInternationalization and localizationPreparing for internationalization and localizationInvestigating tools for internationalization and localizationCICI and the pipelineGetting involved with build management as a testerInvestigating CI toolsRegulated testingNavigating regulatory requirementsSummary
Technical requirementsFinding bugsOracles in software testingInattentional blindness and OraclesAbout the word bugWriting bug reportsEffective bug reportsEffective reproduction stepsPlanning testing – cases and plansTest cases in practiceMetrics and measurementMetric dysfunctionProject projectionsInfluencing changeSummarizing informationSummary
Technical requirementsThe test data problemTest data and database appsThe standard data seedTable-driven – when test data drives behaviorScriptable users and structureExploring synthetic usersLeveraging production refreshesExploring development, test, and production environmentsUnderstanding the regulatory issues in test data managementThe user spy featureSummary

Technical requirementsWaterfallThe advantages of and place for the waterfallThe V-ModelIterative, incremental, and mini waterfallsExtreme Programming (XP)The context of XPScrum and SAFeThe context of ScrumSAFe and its contextHouse-rules software developmentThe Agile ManifestoContext-driven testingIllustrations of the principles in actionKanban as a house ruleContinuous delivery and deploymentDevOps, Platform Engineering, SRESummaryFurther reading
Technical requirementsRecipes – how to do hard thingsDefining recipesShouldn’t recipes just be automated?Do recipes overlap with technical documentation?Coverage – did we test the right things well enough?Precise code coverage measuresClosing out coverageDefects – what is the status of the software?Schedule and risk – too many test ideas, not enough timeIterative testingStrategy – what are our risks and priorities?Dashboard – how do we communicate what we know?Summary
What are we doing now?Getting the form filled outThe elevator pitchA census of riskDefining a real risk censusSetting priorities, time management, and scopeToday’s strategy versus tomorrow’s goalsSummary
Lean software testing definedFrom ideas in practice to the term “Lean”The seven wastesWaste #1 – transportWaste #2 – inventoryWaste #3 – motionWaste #4 – waitingWaste #5 – overprocessingWaste #6 – overproductionWaste #7 – defects(New) waste #8 – abilityRemoving waste in testingFlowVisualizing flow – an exampleMultitaskingMeasurement – lead time versus cycle timeEfficiency and congestionMetric – touch timeBatch sizeQueues and efficiencyArrival time pacingLimiting work in progress to create a pull systemRelease cadenceOne-piece flow and CDSummary
RCRCRC at scaleThe 1-day test planRCRCRC in the enterpriseA coverage dashboardTest coverage blindersPair and tri-programmingDiscovering the expertise paradoxMaking expertise transparentThe evolution of the test strategyThe alternative to evolution – information hidingProfessional pushback – dealing with bulliesPower in the workplaceOne way to say itBoundaries in the enterpriseNarcissistic communicationBoundary enforcement – the power moveIf you choose to staySummary
Technical requirementsThe cultural conflict with a testing roleHow we got here – the bad newsHow we got here – developing at internet speedBuilding a risk mitigation teamThe purpose of the risk exerciseFaith-based versus empirical test automationThe math behind faith-based test automationPossible outcomesShift left and shift right(Actually) continuous testingSummary
Philosophy and why it matters in testingSprint Length: it dependsShu Ha Ri: learning by osmosisA tough question: what do you want?Ethics and ethical reasoning in testingEthical frameworksClassic ethical challengesPractical ethical issues in testingSkipping test steps and the good newsDecisions are not made in meetingsScientific thinking and logical fallaciesLogical fallaciesHow we wind up in hell and how to escapePut the responsibility in the right placeSummaryAdditional readings for the serious philosopher
Context-driven testing and the other schoolsThe Agile schoolThe DevOps or CD schoolThe analytical schoolThe factory schoolThe quality schoolThe context-driven schoolPrecise languageWordsmatterThe benefits of being impreciseProcess versus skillTesting and checkingYes, we can assure qualitySummaryFurther reading
A mobile test strategy exampleOur sample mobile appDesigning a test programMobile system coordination and deploymentThe human elementAI in software testingThe state of AI toolsPossible ways to use AI for testingOther forms of AI in testingThe bottom line for AI testingA few thoughts to leave withThoughts on documentationSummary
Other Books You May EnjoyPackt is searching for authors like youShare your thoughtsDownload a free PDF copy of this book

Content preview from Software Testing Strategies

4 Customer-Facing Tests

In previous chapters, we introduced the idea that testing can be programmer-facing or customer-facing. We also defined testing as a technical investigation into the state of the software. For this chapter, our first problem is likely to be figuring out what the test automation portion of customer-facing test automation means. To accomplish this, we’ll discuss the consequences of the idea and then list some patterns we have seen to make the approach successful.

In this chapter, we cover the following topics: