book

Test-Driven Development with Python, 2nd Edition

by Harry Percival

August 2017

Intermediate to advanced

624 pages

12h 18m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Why I Wrote a Book About Test-Driven DevelopmentAims of This BookOutlineConventions Used in This BookSubmitting ErrataUsing Code ExamplesO’Reilly SafariContacting O’Reilly
Python 3 and ProgrammingHow HTML WorksDjangoJavaScriptRequired Software InstallationsGit’s Default Editor, and Other Basic Git ConfigInstalling Firefox and GeckodriverSetting Up Your VirtualenvActivating and Deactivating the VirtualenvInstalling Django and SeleniumSome Error Messages You’re Likely to See When You Inevitably Fail to Activate Your Virtualenv
Additional Thanks for the Second Edition
Obey the Testing Goat! Do Nothing Until You Have a TestGetting Django Up and RunningStarting a Git Repository
Using a Functional Test to Scope Out a Minimum Viable AppThe Python Standard Library’s unittest ModuleCommit
Our First Django App, and Our First Unit TestUnit Tests, and How They Differ from Functional TestsUnit Testing in DjangoDjango’s MVC, URLs, and View FunctionsAt Last! We Actually Write Some Application Code!urls.pyUnit Testing a ViewThe Unit-Test/Code Cycle
Programming Is Like Pulling a Bucket of Water Up from a WellUsing Selenium to Test User InteractionsThe “Don’t Test Constants” Rule, and Templates to the RescueRefactoring to Use a TemplateThe Django Test ClientOn RefactoringA Little More of Our Front PageRecap: The TDD Process
Wiring Up Our Form to Send a POST RequestProcessing a POST Request on the ServerPassing Python Variables to Be Rendered in the TemplateAn Unexpected FailureThree Strikes and RefactorThe Django ORM and Our First ModelOur First Database MigrationThe Test Gets Surprisingly FarA New Field Means a New MigrationSaving the POST to the DatabaseRedirect After a POSTBetter Unit Testing Practice: Each Test Should Test One ThingRendering Items in the TemplateCreating Our Production Database with migrateRecap

Ensuring Test Isolation in Functional TestsRunning Just the Unit TestsAside: Upgrading Selenium and GeckodriverOn Implicit and Explicit Waits, and Voodoo time.sleeps
Small Design When NecessaryNot Big Design Up FrontYAGNI!REST (ish)Implementing the New Design Incrementally Using TDDEnsuring We Have a Regression TestIterating Towards the New DesignTaking a First, Self-Contained Step: One New URLA New URLA New View FunctionGreen? RefactorAnother Small Step: A Separate Template for Viewing ListsA Third Small Step: A URL for Adding List ItemsA Test Class for New List CreationA URL and View for New List CreationRemoving Now-Redundant Code and TestsA Regression! Pointing Our Forms at the New URLBiting the Bullet: Adjusting Our ModelsA Foreign Key RelationshipAdjusting the Rest of the World to Our New ModelsEach List Should Have Its Own URLCapturing Parameters from URLsAdjusting new_list to the New WorldThe Functional Tests Detect Another RegressionOne More View to Handle Adding Items to an Existing ListBeware of Greedy Regular Expressions!The Last New URLThe Last New ViewTesting the Response Context Objects DirectlyA Final Refactor Using URL includes
What to Functionally Test About Layout and StylePrettification: Using a CSS FrameworkDjango Template InheritanceIntegrating BootstrapRows and ColumnsStatic Files in DjangoSwitching to StaticLiveServerTestCaseUsing Bootstrap Components to Improve the Look of the SiteJumbotron!Large InputsTable StylingUsing Our Own CSSWhat We Glossed Over: collectstatic and Other Static DirectoriesA Few Things That Didn’t Make It
TDD and the Danger Areas of DeploymentAs Always, Start with a TestGetting a Domain NameManually Provisioning a Server to Host Our SiteChoosing Where to Host Our SiteSpinning Up a ServerUser Accounts, SSH, and PrivilegesInstalling Python 3.6Configuring Domains for Staging and LiveDeploying Our Code ManuallyCreating a Virtualenv on the Server Using requirements.txtUsing the FT to Check That Our Deployment WorksDebugging a Deployment That Doesn’t Seem to Work at AllHacking ALLOWED_HOSTS in settings.pyCreating the Database with migrateSuccess! Our Hack Deployment Works
What We Need to DoSwitching to NginxInstallationThe FT Now Fails, But Show Nginx Is RunningSimple Nginx ConfigurationSwitching to GunicornGetting Nginx to Serve Static FilesSwitching to Using Unix SocketsUsing Environment Variables to Adjust Settings for ProductionEssential Googling the Error MessageFixing ALLOWED_HOSTS with Nginx: passing on the Host headerUsing a .env File to Store Our Environment VariablesGenerating a secure SECRET_KEYUsing Systemd to Make Sure Gunicorn Starts on BootSaving Our Changes: Adding Gunicorn to Our requirements.txtThinking About AutomatingSaving Templates for Our Provisioning Config FilesSaving Our Progress
Breakdown of a Fabric Script for Our DeploymentPulling Down Our Source Code with GitUpdating the VirtualenvCreating a New .env File if NecessaryUpdating Static FilesMigrating the Database If NecessaryTrying It OutDeploying to LiveProvisioning: Nginx and Gunicorn Config Using sedGit Tag the ReleaseFurther ReadingAutomating Provisioning with Ansible
Start on a Validation FT: Preventing Blank ItemsSkipping a TestSplitting Functional Tests Out into Many FilesRunning a Single Test FileA New Functional Test Tool: A Generic Explicit Wait HelperFinishing Off the FTRefactoring Unit Tests into Several Files
Model-Layer ValidationThe self.assertRaises Context ManagerA Django Quirk: Model Save Doesn’t Run ValidationSurfacing Model Validation Errors in the ViewChecking That Invalid Input Isn’t Saved to the DatabaseDjango Pattern: Processing POST Requests in the Same View as Renders the FormRefactor: Transferring the new_item Functionality into view_listEnforcing Model Validation in view_listRefactor: Removing Hardcoded URLsThe {% url %} Template TagUsing get_absolute_url for Redirects
Moving Validation Logic into a FormExploring the Forms API with a Unit TestSwitching to a Django ModelFormTesting and Customising Form ValidationUsing the Form in Our ViewsUsing the Form in a View with a GET RequestA Big Find and ReplaceUsing the Form in a View That Takes POST RequestsAdapting the Unit Tests for the new_list ViewUsing the Form in the ViewUsing the Form to Display Errors in the TemplateUsing the Form in the Other ViewA Helper Method for Several Short TestsAn Unexpected Benefit: Free Client-Side Validation from HTML5A Pat on the BackBut Have We Wasted a Lot of Time?Using the Form’s Own Save Method
Another FT for Duplicate ItemsPreventing Duplicates at the Model LayerA Little Digression on Queryset Ordering and String RepresentationsRewriting the Old Model TestSome Integrity Errors Do Show Up on SaveExperimenting with Duplicate Item Validation at the Views LayerA More Complex Form to Handle Uniqueness ValidationUsing the Existing List Item Form in the List ViewWrapping Up: What We’ve Learned About Testing Django
Starting with an FTSetting Up a Basic JavaScript Test RunnerUsing jQuery and the Fixtures DivBuilding a JavaScript Unit Test for Our Desired FunctionalityFixtures, Execution Order, and Global State: Key Challenges of JS Testingconsole.log for Debug PrintingUsing an Initialize Function for More Control Over Execution TimeColumbo Says: Onload Boilerplate and NamespacingJavaScript Testing in the TDD CycleA Few Things That Didn’t Make It
Staging DeployLive DeployWhat to Do If You See a Database ErrorWrap-Up: git tag the New Release
Passwordless AuthExploratory Coding, aka “Spiking”Starting a Branch for the SpikeFrontend Log in UISending Emails from DjangoAnother Secret, Another Environment VariableStoring Tokens in the DatabaseCustom Authentication ModelsFinishing the Custom Django AuthDe-spikingReverting Our Spiked CodeA Minimal Custom User ModelTests as DocumentationA Token Model to Link Emails with a Unique ID
Before We Start: Getting the Basic Plumbing InMocking Manually, aka MonkeypatchingThe Python Mock LibraryUsing unittest.patchGetting the FT a Little Further AlongTesting the Django Messages FrameworkAdding Messages to Our HTMLStarting on the Login URLChecking That We Send the User a Link with a TokenDe-spiking Our Custom Authentication Backend1 if = 1 More TestThe get_user MethodUsing Our Auth Backend in the Login ViewAn Alternative Reason to Use Mocks: Reducing DuplicationUsing mock.return_valuePatching at the Class LevelThe Moment of Truth: Will the FT Pass?It Works in Theory! Does It Work in Practice?Using Our New Environment Variable, and Saving It to .envFinishing Off Our FT, Testing Logout
Skipping the Login Process by Pre-creating a SessionChecking That It WorksOur Final Explicit Wait Helper: A Wait Decorator
The Proof Is in the Pudding: Using Staging to Catch Final BugsInspecting Logs on the ServerAnother Environment VariableAdapting Our FT to Be Able to Test Real Emails via POP3Managing the Test Database on StagingA Django Management Command to Create SessionsGetting the FT to Run the Management Command on the ServerUsing Fabric Directly from PythonRecap: Creating Sessions Locally Versus StagingUpdating our Deploy ScriptWrap-Up
The Alternative: “Inside-Out”Why Prefer “Outside-In”?The FT for “My Lists”The Outside Layer: Presentation and TemplatesMoving Down One Layer to View Functions (the Controller)Another Pass, Outside-InA Quick Restructure of the Template Inheritance HierarchyDesigning Our API Using the TemplateMoving Down to the Next Layer: What the View Passes to the TemplateThe Next “Requirement” from the Views Layer: New Lists Should Record OwnerA Decision Point: Whether to Proceed to the Next Layer with a Failing TestMoving Down to the Model LayerFinal Step: Feeding Through the .name API from the Template
Revisiting Our Decision Point: The Views Layer Depends on Unwritten Models CodeA First Attempt at Using Mocks for IsolationUsing Mock side_effects to Check the Sequence of EventsListen to Your Tests: Ugly Tests Signal a Need to RefactorRewriting Our Tests for the View to Be Fully IsolatedKeep the Old Integrated Test Suite Around as a Sanity CheckA New Test Suite with Full IsolationThinking in Terms of CollaboratorsMoving Down to the Forms LayerKeep Listening to Your Tests: Removing ORM Code from Our ApplicationFinally, Moving Down to the Models LayerBack to ViewsThe Moment of Truth (and the Risks of Mocking)Thinking of Interactions Between Layers as “Contracts”Identifying Implicit ContractsFixing the OversightOne More TestTidy Up: What to Keep from Our Integrated Test SuiteRemoving Redundant Code at the Forms LayerRemoving the Old Implementation of the ViewRemoving Redundant Code at the Forms LayerConclusions: When to Write Isolated Versus Integrated TestsLet Complexity Be Your GuideShould You Do Both?Onwards!
Installing JenkinsConfiguring JenkinsInitial UnlockSuggested Plugins for NowConfiguring the Admin UserAdding PluginsTelling Jenkins Where to Find Python 3 and XvfbFinishing Off with HTTPSSetting Up Our ProjectFirst Build!Setting Up a Virtual Display So the FTs Can Run HeadlessTaking ScreenshotsIf in Doubt, Try Bumping the Timeout!Running Our QUnit JavaScript Tests in Jenkins with PhantomJSInstalling nodeAdding the Build Steps to JenkinsMore Things to Do with a CI Server
An FT with Multiple Users, and addCleanupThe Page PatternExtend the FT to a Second User, and the “My Lists” PageAn Exercise for the Reader
Thesis: Unit Tests Are Superfast and Good Besides ThatFaster Tests Mean Faster DevelopmentThe Holy Flow StateSlow Tests Don’t Get Run as Often, Which Causes Bad CodeWe’re Fine Now, but Integrated Tests Get Slower Over TimeDon’t Take It from MeAnd Unit Tests Drive Good DesignThe Problems with “Pure” Unit TestsIsolated Tests Can Be Harder to Read and WriteIsolated Tests Don’t Automatically Test IntegrationUnit Tests Seldom Catch Unexpected BugsMocky Tests Can Become Closely Tied to ImplementationBut All These Problems Can Be OvercomeSynthesis: What Do We Want from Our Tests, Anyway?CorrectnessClean, Maintainable CodeProductive WorkflowEvaluate Your Tests Against the Benefits You Want from ThemArchitectural SolutionsPorts and Adapters/Hexagonal/Clean ArchitectureFunctional Core, Imperative ShellConclusionFurther Reading
Testing Is HardKeep Your CI Builds GreenTake Pride in Your Tests, as You Do in Your CodeRemember to Tip the Bar StaffDon’t Be a Stranger!
Running Firefox Selenium Sessions with XvfbSetting Up Django as a PythonAnywhere Web AppCleaning Up /tmpScreenshotsThe Deployment Chapter
Class-Based Generic ViewsThe Home Page as a FormViewUsing form_valid to Customise a CreateViewA More Complex View to Handle Both Viewing and Adding to a ListThe Tests Guide Us, for a WhileUntil We’re Left with Trial and ErrorBack on TrackIs That Your Final Answer?Compare Old and NewBest Practices for Unit Testing CBGVs?Take-Home: Having Multiple, Isolated View Tests with Single Assertions Helps
Installing System Packages and NginxConfiguring Gunicorn, and Using Handlers to Restart ServicesWhat to Do NextMove Deployment out of Fabric and into AnsibleUse Vagrant to Spin Up a Local VM
An Attempted Deploy to StagingRunning a Test Migration LocallyEntering Problematic DataCopying Test Data from the Live SiteConfirming the ErrorInserting a Data MigrationRe-creating the Old MigrationTesting the New Migrations TogetherConclusions
What Is BDD?Basic HousekeepingWriting an FT as a “Feature” Using Gherkin SyntaxAs-a /I want to/So thatGiven/When/ThenNot Always a Perfect Fit!Coding the Step FunctionsGenerating Placeholder StepsFirst Step DefinitionsetUp and tearDown Equivalents in environment.pyAnother RunCapturing Parameters in StepsComparing the Inline-Style FTBDD Encourages Structured Test CodeThe Page Pattern as an AlternativeBDD Might Be Less Expressive than Inline CommentsWill Nonprogrammers Write Tests?Some Tentative Conclusions
Our Approach for This AppendixChoosing Our Test ApproachBasic PipingActually Responding with SomethingAdding POSTTesting the Client-Side Ajax with Sinon.jsSinon and Testing the Asynchronous Part of AjaxWiring It All Up in the Template to See If It Really WorksImplementing Ajax POST, Including the CSRF TokenMocking in JavaScriptFinishing the Refactor: Getting the Tests to Match the CodeData Validation: An Exercise for the Reader?
InstallationSerializers (Well, ModelSerializers, Really)Viewsets (Well, ModelViewsets, Really) and RoutersA Different URL for POST ItemAdapting the Client SideWhat Django-Rest-Framework Gives YouConfiguration Instead of CodeFree Functionality
Initial Project SetupThe Basic TDD WorkflowMoving Beyond Dev-Only TestingGeneral Testing Best PracticesSelenium/Functional Testing Best PracticesOutside-In, Test Isolation Versus Integrated Tests, and Mocking
Notifications—Both on the Site and by EmailSwitch to PostgresRun Your Tests Against Different Browsers404 and 500 TestsThe Django Admin SiteWrite Some Security TestsTest for Graceful DegradationCaching and Performance TestingJavaScript MVC FrameworksAsync and WebsocketsSwitch to Using py.testCheck Out coverage.pyClient-Side EncryptionYour Suggestion Here
Full List of Links for Each ChapterUsing Git to Check Your ProgressDownloading a ZIP File for a ChapterDon’t Let it Become a Crutch!

Content preview from Test-Driven Development with Python, 2nd Edition

Chapter 24. Continuous Integration (CI)

As our site grows, it takes longer and longer to run all of our functional tests. If this continues, the danger is that we’re going to stop bothering.

Rather than let that happen, we can automate the running of functional tests by setting up a “Continuous Integration” or CI server. That way, in day-to-day development, we can just run the FT that we’re working on at that time, and rely on the CI server to run all the tests automatically and let us know if we’ve broken anything accidentally. The unit tests should stay fast enough that we can keep running them every few seconds.

The CI server of choice these days is called Jenkins. It’s a bit Java, a bit crashy, a bit ugly, but it’s what everyone uses, and it has a great plugin ecosystem, so let’s get it up and running.

Installing Jenkins

There are several hosted-CI services out there that essentially provide you with a Jenkins server, ready to go. I’ve come across Sauce Labs, Travis, Circle-CI, ShiningPanda, and there are probably lots more. But I’m going to assume we’re installing everything on a server we control.

Note

It’s not a good idea to install Jenkins on the same server as our staging or production servers. Apart from anything else, we may want Jenkins to be able to reboot the staging server!

We’ll install the latest version from the official Jenkins apt repo, because the Ubuntu default still has a few annoying bugs with locale/unicode support, and it also doesn’t set itself up ...