O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Python Testing Cookbook.

Book Description

Fix everyday testing problems in Python with the help of this solution-based guide

About This Book
  • Use powerful tools such as doctest and unittest to make testing convenient
  • Apply automation testing to an existing legacy system that isn't test oriented
  • A practical guide to ease testing in Python using real-world examples
Who This Book Is For

If you're a Python developer who wants to take testing to the next level and would like to expand your testing skills, this book is for you. It is assumed that you have some Python programming knowledge.

What You Will Learn
  • Run test cases from the command line with increased verbosity
  • Write a Nose extension to pick tests based on regular expressions
  • Create testable documentation using doctest
  • Use Selenium to test the Web User Interface
  • Write a testable story with Voidspace Mock and Nose
  • Configure TeamCity to run Python tests on commit
  • Update project-level scripts to provide coverage reports
In Detail

Automated testing is the best way to increase efficiency while reducing the defects of software testing. It helps find bugs in code easily and at an early stage so that they can be tackled efficiently. This book delves into essential testing concepts used in Python to help you build robust and maintainable code.

Python Testing Cookbook begins with a brief introduction to Python's unit testing framework to help you write automated test cases. You will learn how to write suitable test sets for your software and run automated test suites with Nose. You will then work with the unittest.mock library, which allows you to replace the parts of your system that are being tested with mock objects and make assertions about how they have been used. You will also see how to apply Test-driven Development (TDD) and Behavior-driven Development (BDD) and how to eliminate issues caused by TDD. The book explains how to integrate automated tests using Continuous Integration and perform smoke/load testing. It also covers best practices and will help you solve persistent testing issues in Python. The book concludes by helping you understand how doctest works and how Selenium can be used to test code efficiently.

Style and approach

A solution-based approach consisting of over 50 recipes to ease testing Python code.

Downloading the example code for this book You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Table of Contents

  1. Title Page
  2. Copyright and Credits
    1. Python Testing Cookbook Second Edition
  3. Packt Upsell
    1. Why subscribe?
    2. PacktPub.com
  4. Contributors
    1. About the authors
    2. About the reviewers
    3. Packt is searching for authors like you
  5. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Conventions used
    4. Sections
      1. Getting ready
      2. How to do it…
      3. How it works…
      4. There's more…
      5. See also
    5. Get in touch
      1. Reviews
  6. Using Unittest to Develop Basic Tests
    1. Introduction
    2. Asserting the basics
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. assertEquals is preferred over assertTrue and assertFalse
        2. self.fail([msg]) can usually be rewritten with assertions
        3. Our version of Python can impact our options
    3. Setting up and tearing down a test harness
      1. How to do it...
      2. How it works...
    4. Running test cases from the command line
      1. How to do it...
      2. How it works...
      3. There's more...
    5. Running a subset of test case methods
      1. How to do it...
      2. How it works...
    6. Chaining together a suite of tests
      1. How to do it...
      2. How it works...
      3. There's more...
        1. The name of the test case should be significant
    7. Defining test suites inside the test module
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Test suite methods must be outside of the test class
        2. Why have different suites?
        3. optparse is being phased out and replaced by argparse
    8. Retooling old test code to run inside unittest
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Where are the bugs?
        2. FunctionTestCase is a temporary measure
    9. Breaking down obscure tests into simple ones
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Where is the bug?
        2. What is the right size for a test method?
        3. Unittests versus integration tests
    10. Testing the edges
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Identifying the edges is important
        2. Testing for unexpected conditions
    11. Testing corner cases by iteration
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Does this defy the recipe – breaking down obscure tests into simple ones?
        2. How does this compare with the recipe – testing the edges?
      4. See also
  7. Running Automated Test Suites with Nose
    1. Introduction
    2. Getting nosy with testing
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Nose is extensible
        2. Nose is embeddable
      4. See also
    3. Embedding nose inside Python
      1. How to do it...
      2. How it works...
      3. There's more...
    4. Writing a nose extension to pick tests based on regular expressions
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
    5. Writing a nose extension to generate a CSV report
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
    6. Writing a project-level script that lets you run different test suites
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Why use getopt instead of optparse?
  8. Creating Testable Documentation with doctest
    1. Introduction
    2. Documenting the basics
      1. How to do it...
      2. How it works...
      3. There's more...
    3. Catching stack traces
      1. How to do it...
      2. How it works...
    4. Running a doctest from the command line
      1. How to do it...
      2. How it works...
    5. Coding a test harness for doctest
      1. How to do it...
      2. How it works...
      3. There's more...
    6. Filtering out test noise
      1. How to do it...
      2. How it works...
      3. There's more...
    7. Printing out all your documentation including a status report
      1. How to do it...
      2. How it works...
      3. There's more...
    8. Testing the edges
      1. How to do it...
      2. How it works...
      3. There's more...
      4. See also
    9. Testing corner cases by iteration
      1. How to do it...
      2. How it works...
        1. Does this type of test fit better into doctest or unittest?
      3. See also
    10. Getting nosy with doctest
      1. Getting ready
      2. How to do it...
      3. How it works...
    11. Updating the project-level script to run this chapter's doctests
      1. How to do it...
      2. How it works...
      3. There's more...
  9. Testing Customer Stories with Behavior-Driven Development
    1. Introduction
    2. Naming tests that sound like sentences and stories
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
    3. Testing separate doctest documents
      1. Getting ready 
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Doesn't this defy the usability of docstrings?
    4. Writing a testable story with doctest
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
    5. Writing a testable novel with doctest
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
    6. Writing a testable story with Voidspace Mock and nose
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Tell me more about the spec nose plugin!
        2. Why didn't we reuse the plugin from the recipe "Naming tests so they sound like sentences and stories"?
      5. See also
    7. Writing a testable story with mockito and nose
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    8. Writing a testable story with Lettuce
      1. Getting ready...
      2. How to do it...
      3. How it works...
      4. There's more...
      5. How complex should a story be?
        1. Don't mix wiring code with application code
        2. Lettuce works great using folders
      6. See also
    9. Using Should DSL to write succinct assertions with Lettuce
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
      5. See also
    10. Updating the project-level script to run this chapter's BDD tests
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
  10. High-Level Customer Scenarios with Acceptance Testing
    1. Introduction
    2. Installing Pyccuracy
      1. How to do it...
      2. How it works...
      3. See also
    3. Testing the basics with Pyccuracy
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
      5. See also
    4. Using Pyccuracy to verify web app security
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
    5. Installing Robot Framework
      1. How to do it...
      2. There's more...
    6. Creating a data-driven test suite with Robot Framework
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Do I have to write HTML tables?
        2. What are the best ways to write the code that implements our custom keywords?
        3. Robot Framework variables are Unicode
      5. See also
    7. Writing a testable story with Robot Framework
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Given-When-Then results in duplicate rules
        2. Do the try-except blocks violate the idea of keeping things light?
      5. See also
    8. Tagging Robot Framework tests and running a subset
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. What about documentation?
      5. See also
    9. Testing web basics with Robot Framework
      1. Getting ready...
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Learn about timing configurations – they may be important!
      5. See also
    10. Using Robot Framework to verify web app security
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Why not use a "remember me" option?
        2. Shouldn't we refactor the first test scenario to use the keyword?
        3. Would arguments make the login keyword more flexible?
      5. See also
    11. Creating a project-level script to verify this chapter's acceptance tests
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more
        1. Can we only use getopt?
        2. What's wrong with using the various command-line tools?
  11. Integrating Automated Tests with Continuous Integration
    1. Introduction
    2. Generating a CI report for Jenkins using NoseXUnit
      1. Getting ready
      2. How to do it...
      3. How it works...
    3. Configuring Jenkins to run Python tests upon commit
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Do I have to use git for source code management?
        2. What is the format of polling?
      5. See also
    4. Configuring Jenkins to run Python tests when scheduled
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Jenkins versus TeamCity
      5. See also
    5. Generating a CI report for TeamCity using teamcity-nose
      1. Getting ready
      2. How to do it...
      3. How it works...
    6. Configuring TeamCity to run Python tests upon commit
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. What did teamcity-nose give us?
      5. See also
    7. Configuring TeamCity to run Python tests when scheduled
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. See also
  12. Measuring Your Success with Test Coverage
    1. Introduction
    2. Building a network management application
      1. How to do it...
      2. How it works...
    3. Installing and running coverage on your test suite
      1. How to do it...
      2. There's more...
        1. Why are there no asserts in unittest?
    4. Generating an HTML report using coverage
      1. How to do it...
      2. How it works...
    5. Generating an XML report using coverage
      1. How to do it...
      2. How it works...
        1. What use is an XML report?
      3. See also
    6. Getting nosy with coverage
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Why use the nose plugin instead of the coverage tool directly?
        2. Why are SQLite3 and Spring Python included?
    7. Filtering out test noise from coverage
      1. How to do it...
      2. How it works...
      3. There's more...
      4. See also
    8. Letting Jenkins get nosy with coverage
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Nose doesn't directly support coverage's XML option
    9. Updating the project-level script to provide coverage reports
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Can we only use getopt?
  13. Smoke/Load Testing – Testing Major Parts
    1. Introduction
    2. Defining a subset of test cases using import statements
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Security, checking, and integration aren't smoke tests!
        2. What provides good flexibility?
      4. See also
    3. Leaving out integration tests
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Should a smoke test include integration or unit tests?
      4. See also
    4. Targeting end-to-end scenarios
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. How does this define smoke tests?
      5. See also
    5. Targeting the test server
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. How likely is it that a development and production environment would use two different database systems?
        2. This isn't just confined to database systems
    6. Coding a data simulator
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. Why does the server script initialize the database?
        2. Why MySQL instead of SQLite?
      5. See also
    7. Recording and playing back live data in real time
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. I thought this recipe was about live data!
        2. Is opening and closing a file for every event a good idea?
        3. What about offloading the storage of data?
      5. See also
    8. Recording and playing back live data as fast as possible
      1. Getting ready
      2. How to do it...
      3. How it works...
      4. There's more...
        1. What is the difference between this and playing back in real time?
        2. Where are the breaking points of this application?
        3. What amount of live data should be collected?
      5. See also
    9. Automating your management demo
      1. How to do it...
      2. How it works...
      3. There's more...
        1. What if my manager likes to take detours?
  14. Good Test Habits for New and Legacy Systems
    1. Introduction
    2. Something is better than nothing
      1. How to do it...
      2. How it works...
        1. That can't be everything!
      3. See also
    3. Coverage isn't everything
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Are we not supposed to increase coverage?
        2. But I want to brag about the coverage of my system!
    4. Be willing to invest in test fixtures
      1. How to do it....
      2. How it works...
      3. There's more...
        1. Is this just about setting up a database?
    5. If you aren't convinced about the value of testing, your team won't be either
    6. Harvesting metrics
      1. How to do it...
      2. How it works...
      3. There's more...
        1. Metrics aren't just for defending yourself to management
    7. Capturing a bug in an automated test
      1. How to do it...
      2. How it works...
      3. There's more...
        1. When the time comes to add a completely new module, you will be ready for it
        2. Don't give into the temptation to skip testing
    8. Separating algorithms from concurrency
      1. How to do it...
      2. How it works...
        1. Research test options provided by your concurrency frameworks
    9. Pause to refactor when a test suite takes too long to run
      1. How to do it...
      2. How it works...
      3. There's more...
      4. See also
    10. Cash in on your confidence
      1. How to do it...
      2. How it works...
    11. Be willing to throw away an entire day's changes
      1. How to do it...
      2. How it works...
      3. There's more...
        1. How does this mesh with "Something is better than nothing"
      4. See also
    12. Instead of shooting for 100 percent coverage, try to have a steady growth
      1. How to do it...
      2. How it works...
    13. Randomly breaking up your app can lead to better code
      1. How to do it...
      2. How it works...
      3. There's more...
        1. How does this compare to fuzz testing?
        2. Are there any tools to help with this?
  15. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think