Mastering Object-oriented Python

Mastering Object-oriented Python

by Steven Lott
Released April 2014
Publisher(s): Packt Publishing
ISBN: 9781783280971

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

For those who know the basics of object-oriented Python this book is a must-have. With 750 code samples and a relaxed tutorial approach, it's the seamless route to more sophisticated programming.

In Detail

This practical example-oriented guide will teach you advanced concepts of object-oriented programming in Python. This book will present detailed examples of almost all of the special method names that support creating classes that integrate seamlessly with Python's built-in features. It will show you how to use JSON, YAML, Pickle, CSV, XML, Shelve, and SQL to create persistent objects and transmit objects between processes. The book also covers logging, warnings, unit testing, configuration files, and how to work with the command line.

This book is broken into three major parts: Pythonic Classes via Special Methods; Persistence and Serialization; Testing, Debugging, Deploying, and Maintaining. The special methods are broken down into several focus areas: initialization, basics, attribute access, callables, contexts, containers, collections, numbers, and more advanced techniques such as decorators and mixin classes.

What You Will Learn

  • Understand the different design patterns for the __init__() method
  • Discover the essential features of Python 3's abstract base classes and how you can use them for your own applications
  • Design callable objects and context managers that leverage the with statement
  • Perform object serialization in formats such as JSON, YAML, Pickle, CSV, and XML
  • Employ the Shelve module as a sophisticated local database
  • Map Python objects to a SQL database using the built-in SQLite module
  • Transmit Python objects via RESTful web services
  • Devise strategies for automated unit testing, including how to use the doctest and the unittest.mock module
  • Parse command-line arguments and integrate this with configuration files and environment variables

Publisher resources

Download Example Code

Table of contents

  1. Mastering Object-oriented Python
    1. Table of Contents
    2. Mastering Object-oriented Python
    3. Credits
    4. About the Author
    5. About the Reviewers
    6. www.PacktPub.com
      1. Support files, eBooks, discount offers and more
        1. Why Subscribe?
        2. Free Access for Packt account holders
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
        1. Customer support
        2. Downloading the example code for this book
        3. Errata
        4. Piracy
        5. Questions
    8. Some Preliminaries
      1. About casino Blackjack
        1. Playing the game
        2. Blackjack player strategies
        3. Object design for simulating Blackjack
      2. Performance – the timeit module
      3. Testing – unittest and doctest
        1. Unit testing and technology spikes
      4. Docstrings – RST markup and documentation tools
      5. The IDE question
      6. About special method names
      7. Summary
    9. 1. Pythonic Classes via Special Methods
      1. Pythonic Classes via Special Methods
      2. 1. The __init__() Method
        1. The implicit superclass – object
        2. The base class object __init__() method
        3. Implementing __init__() in a superclass
        4. Using __init__() to create manifest constants
        5. Leveraging __init__() via a factory function
          1. Faulty factory design and the vague else clause
          2. Simplicity and consistency using elif sequences
          3. Simplicity using mapping and class objects
            1. Two parallel mappings
            2. Mapping to a tuple of values
            3. The partial function solution
            4. Fluent APIs for factories
        6. Implementing __init__() in each subclass
        7. Simple composite objects
          1. Wrapping a collection class
          2. Extending a collection class
          3. More requirements and another design
        8. Complex composite objects
          1. Complete composite object initialization
        9. Stateless objects without __init__()
        10. Some additional class definitions
        11. Multi-strategy __init__()
          1. More complex initialization alternatives
          2. Initializing static methods
        12. Yet more __init__() techniques
          1. Initialization with type validation
          2. Initialization, encapsulation, and privacy
        13. Summary
      3. 2. Integrating Seamlessly with Python Basic Special Methods
        1. The __repr__() and __str__() methods
          1. Non collection __str__() and __repr__()
          2. Collection __str__() and __repr__()
        2. The __format__() method
          1. Nested formatting specifications
          2. Collections and delegating format specifications
        3. The __hash__() method
          1. Deciding what to hash
          2. Inheriting definitions for immutable objects
          3. Overriding definitions for immutable objects
          4. Overriding definitions for mutable objects
          5. Making a frozen hand from a mutable hand
        4. The __bool__() method
        5. The __bytes__() method
        6. The comparison operator methods
          1. Designing comparisons
          2. Implementation of comparison for objects of the same class
          3. Implementation of comparison for objects of mixed classes
          4. Hard totals, soft totals, and polymorphism
          5. A mixed class comparison example
        7. The __del__() method
          1. The reference count and destruction
          2. Circular references and garbage collection
          3. Circular references and the weakref module
          4. The __del__() and close() methods
        8. The __new__() method and immutable objects
        9. The __new__() method and metaclasses
          1. Metaclass example 1 – ordered attributes
          2. Metaclass example 2 – self-reference
        10. Summary
      4. 3. Attribute Access, Properties, and Descriptors
        1. Basic attribute processing
          1. Attributes and the __init__() method
        2. Creating properties
          1. Eagerly computed properties
          2. Setter and deleter properties
        3. Using special methods for attribute access
          1. Creating immutable objects with __slots__
          2. Creating immutable objects as a tuple subclass
          3. Eagerly computed attributes
        4. The __getattribute__() method
        5. Creating descriptors
          1. Using a nondata descriptor
          2. Using a data descriptor
        6. Summary, design considerations, and trade-offs
          1. Properties versus attributes
          2. Designing with descriptors
          3. Looking forward
      5. 4. The ABCs of Consistent Design
        1. Abstract base classes
        2. Base classes and polymorphism
        3. Callables
        4. Containers and collections
        5. Numbers
        6. Some additional abstractions
          1. The iterator abstraction
          2. Contexts and context managers
        7. The abc module
        8. Summary, design considerations, and trade-offs
          1. Looking forward
      6. 5. Using Callables and Contexts
        1. Designing with ABC callables
        2. Improving performance
          1. Using memoization or caching
        3. Using functools for memoization
          1. Aiming for simplicity using the callable API
        4. Complexities and the callable API
        5. Managing contexts and the with statement
          1. Using the decimal context
          2. Other contexts
        6. Defining the __enter__() and __exit__() methods
          1. Handling exceptions
        7. Context manager as a factory
          1. Cleaning up in a context manager
        8. Summary
          1. Callable design considerations and trade-offs
          2. Context manager design considerations and trade-offs
          3. Looking forward
      7. 6. Creating Containers and Collections
        1. ABCs of collections
        2. Examples of special methods
        3. Using the standard library extensions
          1. The namedtuple() function
          2. The deque class
          3. The ChainMap use case
          4. The OrderedDict collection
          5. The defaultdict subclass
          6. The counter collection
        4. Creating new kinds of collections
        5. Defining a new kind of sequence
          1. A statistical list
          2. Choosing eager versus lazy calculation
          3. Working with __getitem__(), __setitem__(), __delitem__(), and slices
          4. Implementing __getitem__(), __setitem__(), and __delitem__()
          5. Wrapping a list and delegating
          6. Creating iterators with __iter__()
        6. Creating a new kind of mapping
        7. Creating a new kind of set
          1. Some design rationale
          2. Defining the Tree class
          3. Defining the TreeNode class
          4. Demonstrating the binary tree set
        8. Summary
          1. Design considerations and Trade-offs
          2. Looking forward
      8. 7. Creating Numbers
        1. ABCs of numbers
          1. Deciding which types to use
          2. The method resolution and the reflected operator concept
        2. The arithmetic operator's special methods
        3. Creating a numeric class
          1. Defining FixedPoint initialization
          2. Defining FixedPoint binary arithmetic operators
          3. Defining FixedPoint unary arithmetic operators
          4. Implementing FixedPoint reflected operators
          5. Implementing FixedPoint comparison operators
        4. Computing a numeric hash
          1. Designing more useful rounding
        5. Implementing other special methods
        6. Optimization with the in-place operators
        7. Summary
          1. Design considerations and trade-offs
          2. Looking forward
      9. 8. Decorators and Mixins – Cross-cutting Aspects
        1. Class and meaning
          1. Constructing the functions
          2. Constructing the class
          3. Some class design principles
          4. Aspect-oriented programming
        2. Using built-in decorators
          1. Using standard library decorators
        3. Using standard library mixin classes
          1. Using the context manager mixin class
          2. Turning off a class feature
        4. Writing a simple function decorator
          1. Creating separate loggers
        5. Parameterizing a decorator
        6. Creating a method function decorator
        7. Creating a class decorator
        8. Adding method functions to a class
        9. Using decorators for security
        10. Summary
          1. Design considerations and trade-offs
          2. Looking forward
    10. 2. Persistence and Serialization
      1. Persistence and Serialization
      2. 9. Serializing and Saving – JSON, YAML, Pickle, CSV, and XML
        1. Understanding persistence, class, state, and representation
          1. Common Python terminologies
        2. Filesystem and network considerations
        3. Defining classes to support persistence
          1. Rendering a blog and posts
        4. Dumping and loading with JSON
          1. Supporting JSON in our classes
          2. Customizing JSON encoding
          3. Customizing JSON decoding
          4. The security and the eval() issue
          5. Refactoring the encode function
          6. Standardizing the date string
          7. Writing JSON to a file
        5. Dumping and loading with YAML
          1. Formatting YAML data on a file
          2. Extending the YAML representation
          3. Security and safe loading
        6. Dumping and loading with pickle
          1. Designing a class for reliable pickle processing
          2. Security and the global issue
        7. Dumping and loading with CSV
          1. Dumping simple sequences to CSV
          2. Loading simple sequences from CSV
          3. Handling containers and complex classes
          4. Dumping and loading multiple row types in a CSV file
          5. Filtering CSV rows with an iterator
          6. Dumping and loading joined rows in a CSV file
        8. Dumping and loading with XML
          1. Dumping objects using string templates
          2. Dumping objects with xml.etree.ElementTree
          3. Loading XML documents
        9. Summary
          1. Design considerations and trade-offs
          2. Schema evolution
          3. Looking forward
      3. 10. Storing and Retrieving Objects via Shelve
        1. Analyzing persistent object use cases
          1. The ACID properties
        2. Creating a shelf
        3. Designing shelvable objects
          1. Designing keys for our objects
          2. Generating surrogate keys for objects
          3. Designing a class with a simple key
          4. Designing classes for containers or collections
          5. Referring to objects via foreign keys
          6. Designing CRUD operations for complex objects
        4. Searching, scanning, and querying
        5. Designing an access layer for shelve
          1. Writing a demonstration script
        6. Creating indexes to improve efficiency
          1. Creating top-level indices
        7. Adding yet more index maintenance
        8. The writeback alternative to index updates
          1. Schema evolution
        9. Summary
          1. Design considerations and trade-offs
          2. Application software layers
          3. Looking forward
      4. 11. Storing and Retrieving Objects via SQLite
        1. SQL databases, persistence, and objects
          1. The SQL data model – rows and tables
          2. CRUD processing via SQL DML statements
          3. Querying rows with the SQL SELECT statement
          4. SQL transactions and the ACID properties
          5. Designing primary and foreign database keys
        2. Processing application data with SQL
          1. Implementing class-like processing in pure SQL
        3. Mapping Python objects to SQLite BLOB columns
        4. Mapping Python objects to database rows manually
          1. Designing an access layer for SQLite
          2. Implementing container relationships
        5. Improving performance with indices
        6. Adding an ORM layer
          1. Designing ORM-friendly classes
          2. Building the schema with the ORM layer
          3. Manipulating objects with the ORM layer
        7. Querying post objects given a tag string
        8. Improving performance with indices
          1. Schema evolution
        9. Summary
          1. Design considerations and trade-offs
          2. Mapping alternatives
          3. Keys and key designs
          4. Application software layers
          5. Looking forward
      5. 12. Transmitting and Sharing Objects
        1. Class, state, and representation
        2. Using HTTP and REST to transmit objects
          1. Implementing CRUD operations via REST
          2. Implementing non-CRUD operations
          3. The REST protocol and ACID
          4. Choosing a representation – JSON, XML, or YAML
        3. Implementing a REST server – WSGI and mod_wsgi
          1. Creating a simple REST application and server
          2. Implementing a REST client
          3. Demonstrating and unit testing the RESTful services
        4. Using Callable classes for WSGI applications
          1. Designing RESTful object identifiers
          2. Multiple layers of REST services
          3. Creating the roulette server
          4. Creating the roulette client
        5. Creating a secure REST service
          1. The WSGI Authentication application
        6. Implementing REST with a web application framework
        7. Using a message queue to transmit objects
          1. Defining processes
          2. Building queues and supplying data
        8. Summary
          1. Design considerations and trade-offs
          2. Schema evolution
          3. Application software layers
          4. Looking forward
      6. 13. Configuration Files and Persistence
        1. Configuration file use cases
        2. Representation, persistence, state, and usability
          1. Application configuration design patterns
          2. Configuring via object construction
          3. Implementing a configuration hierarchy
        3. Storing the configuration in the INI files
        4. Handling more literals via the eval() variants
        5. Storing the configuration in PY files
          1. Configuration via class definitions
          2. Configuration via SimpleNamespace
          3. Using Python with exec() for the configuration
        6. Why is exec() a nonproblem?
        7. Using ChainMap for defaults and overrides
        8. Storing the configuration in JSON or YAML files
          1. Using flattened JSON configurations
          2. Loading a YAML configuration
        9. Storing the configuration in property files
          1. Parsing a properties file
          2. Using a properties file
        10. Storing the configuration in XML files – PLIST and others
          1. Customized XML configuration files
        11. Summary
          1. Design considerations and trade-offs
          2. Creating a shared configuration
          3. Schema evolution
          4. Looking Forward
    11. 3. Testing, Debugging, Deploying, and Maintaining
      1. Testing, Debugging, Deploying, and Maintaining
      2. 14. The Logging and Warning Modules
        1. Creating a basic log
          1. Creating a shared class-level logger
          2. Configuring the loggers
          3. Starting up and shutting down the logging system
          4. Naming the loggers
          5. Extending the logger levels
          6. Defining handlers for multiple destinations
          7. Managing the propagation rules
        2. Configuration gotcha
        3. Specializing logging for control, debug, audit, and security
          1. Creating a debugging log
          2. Creating audit and security logs
        4. Using the warnings module
          1. Showing API changes with a warning
          2. Showing configuration problems with a warning
          3. Showing possible software problems with a warning
        5. Advanced logging – the last few messages and network destinations
          1. Building an automatic tail buffer
          2. Sending logging messages to a remote process
          3. Preventing queue overrun
        6. Summary
          1. Design considerations and trade-offs
          2. Looking forward
      3. 15. Designing for Testability
        1. Defining and isolating units for testing
          1. Minimizing the dependencies
          2. Creating simple unit tests
          3. Creating a test suite
          4. Including edge and corner cases
          5. Mocking dependencies for testing
          6. Using more mocks to test more behaviors
        2. Using doctest to define test cases
          1. Combining doctest and unittest
          2. Creating a more complete test package
        3. Using setup and teardown
          1. Using setup and teardown with OS resources
          2. Using setup and teardown with databases
        4. The TestCase class hierarchy
        5. Using externally defined expected results
        6. Automated integration or performance testing
        7. Summary
          1. Design considerations and trade-offs
          2. Looking forward
      4. 16. Coping With the Command Line
        1. The OS interface and the command line
          1. Arguments and options
        2. Parsing the command line with argparse
          1. A simple on/off option
          2. An option with an argument
          3. Positional arguments
          4. All other arguments
          5. --version display and exit
          6. --help display and exit
        3. Integrating command-line options and environment variables
          1. Providing more configurable defaults
          2. Overriding configuration file settings with environment variables
          3. Overriding environment variables with the configuration files
          4. Making the configuration aware of the None values
        4. Customizing the help output
        5. Creating a top-level main() function
          1. Ensuring DRY for the configuration
          2. Managing nested configuration contexts
        6. Programming In The Large
          1. Designing command classes
          2. Adding the analysis command subclass
          3. Adding and packaging more features into an application
          4. Designing a higher-level composite command
        7. Additional composite command design patterns
        8. Integrating with other applications
        9. Summary
          1. Design considerations and trade-offs
          2. Looking forward
      5. 17. The Module and Package Design
        1. Designing a module
          1. Some module design patterns
          2. Module versus class
          3. The expected content of a module
        2. Whole module versus module items
        3. Designing a package
          1. Designing a module-package hybrid
          2. Designing a package with alternate implementations
        4. Designing a main script and the __main__ module
          1. Creating an executable script file
          2. Creating a __main__ module
          3. Programming in the large
        5. Designing long-running applications
        6. Organizing code into src, bin, and test
        7. Installing Python modules
        8. Summary
          1. Design considerations and trade-offs
          2. Looking forward
      6. 18. Quality and Documentation
        1. Writing docstrings for the help() function
        2. Using pydoc for documentation
        3. Better output via the RST markup
          1. Blocks of text
          2. The RST inline markup
          3. RST directives
          4. Learning RST
        4. Writing effective docstrings
        5. Writing file-level docstrings, including modules and packages
          1. Writing API details in RST markup
          2. Writing class and method function docstrings
          3. Writing function docstrings
        6. More sophisticated markup techniques
        7. Using Sphinx to produce the documentation
          1. Using the Sphinx quickstart
          2. Writing the Sphinx documentation
          3. Filling in the 4+1 views for documentation
          4. Writing the implementation document
          5. Creating the Sphinx cross-references
          6. Refactoring Sphinx files into directories
        8. Writing the documentation
        9. Literate programming
          1. Use cases for literate programming
          2. Working with a literate programming tool
        10. Summary
          1. Design considerations and trade-offs
    12. Index

Product information

  • Title: Mastering Object-oriented Python
  • Author(s): Steven Lott
  • Release date: April 2014
  • Publisher(s): Packt Publishing
  • ISBN: 9781783280971