book

Mastering Object-oriented Python

Name: Mastering Object-oriented Python
Author: Steven F. Lott
ISBN: 9781783280971

by Steven F. Lott

April 2014

Beginner to intermediate

634 pages

15h 22m

English

Packt Publishing

Read now

Unlock full access

Mastering Object-oriented Python
Table of Contents
Mastering Object-oriented Python
Credits
About the Author
About the Reviewers
www.PacktPub.com
Support files, eBooks, discount offers and moreWhy Subscribe?Free Access for Packt account holders
Preface
What this book covers
What you need for this book
Who this book is for

Conventions
Reader feedback
Customer supportDownloading the example code for this bookErrataPiracyQuestions
Some Preliminaries
About casino BlackjackPlaying the gameBlackjack player strategiesObject design for simulating Blackjack
Performance – the timeit module
Testing – unittest and doctest
Unit testing and technology spikes
Docstrings – RST markup and documentation tools
The IDE question
About special method names
Summary
1. Pythonic Classes via Special Methods
Pythonic Classes via Special Methods
1. The __init__() Method
The implicit superclass – object
The base class object __init__() method
Implementing __init__() in a superclass
Using __init__() to create manifest constants
Leveraging __init__() via a factory function
Faulty factory design and the vague else clauseSimplicity and consistency using elif sequencesSimplicity using mapping and class objectsTwo parallel mappingsMapping to a tuple of valuesThe partial function solutionFluent APIs for factories
Implementing __init__() in each subclass
Simple composite objects
Wrapping a collection classExtending a collection classMore requirements and another design
Complex composite objects
Complete composite object initialization
Stateless objects without __init__()
Some additional class definitions
Multi-strategy __init__()
More complex initialization alternativesInitializing static methods
Yet more __init__() techniques
Initialization with type validationInitialization, encapsulation, and privacy
Summary
2. Integrating Seamlessly with Python Basic Special Methods
The __repr__() and __str__() methodsNon collection __str__() and __repr__()Collection __str__() and __repr__()
The __format__() method
Nested formatting specificationsCollections and delegating format specifications
The __hash__() method
Deciding what to hashInheriting definitions for immutable objectsOverriding definitions for immutable objectsOverriding definitions for mutable objectsMaking a frozen hand from a mutable hand
The __bool__() method
The __bytes__() method
The comparison operator methods
Designing comparisonsImplementation of comparison for objects of the same classImplementation of comparison for objects of mixed classesHard totals, soft totals, and polymorphismA mixed class comparison example
The __del__() method
The reference count and destructionCircular references and garbage collectionCircular references and the weakref moduleThe __del__() and close() methods
The __new__() method and immutable objects
The __new__() method and metaclasses
Metaclass example 1 – ordered attributesMetaclass example 2 – self-reference
Summary
3. Attribute Access, Properties, and Descriptors
Basic attribute processingAttributes and the __init__() method
Creating properties
Eagerly computed propertiesSetter and deleter properties
Using special methods for attribute access
Creating immutable objects with __slots__Creating immutable objects as a tuple subclassEagerly computed attributes
The __getattribute__() method
Creating descriptors
Using a nondata descriptorUsing a data descriptor
Summary, design considerations, and trade-offs
Properties versus attributesDesigning with descriptorsLooking forward
4. The ABCs of Consistent Design
Abstract base classes
Base classes and polymorphism
Callables
Containers and collections
Numbers
Some additional abstractions
The iterator abstractionContexts and context managers
The abc module
Summary, design considerations, and trade-offs
Looking forward
5. Using Callables and Contexts
Designing with ABC callables
Improving performance
Using memoization or caching
Using functools for memoization
Aiming for simplicity using the callable API
Complexities and the callable API
Managing contexts and the with statement
Using the decimal contextOther contexts
Defining the __enter__() and __exit__() methods
Handling exceptions
Context manager as a factory
Cleaning up in a context manager
Summary
Callable design considerations and trade-offsContext manager design considerations and trade-offsLooking forward
6. Creating Containers and Collections
ABCs of collections
Examples of special methods
Using the standard library extensions
The namedtuple() functionThe deque classThe ChainMap use caseThe OrderedDict collectionThe defaultdict subclassThe counter collection
Creating new kinds of collections
Defining a new kind of sequence
A statistical listChoosing eager versus lazy calculationWorking with __getitem__(), __setitem__(), __delitem__(), and slicesImplementing __getitem__(), __setitem__(), and __delitem__()Wrapping a list and delegatingCreating iterators with __iter__()
Creating a new kind of mapping
Creating a new kind of set
Some design rationaleDefining the Tree classDefining the TreeNode classDemonstrating the binary tree set
Summary
Design considerations and Trade-offsLooking forward
7. Creating Numbers
ABCs of numbersDeciding which types to useThe method resolution and the reflected operator concept
The arithmetic operator's special methods
Creating a numeric class
Defining FixedPoint initializationDefining FixedPoint binary arithmetic operatorsDefining FixedPoint unary arithmetic operatorsImplementing FixedPoint reflected operatorsImplementing FixedPoint comparison operators
Computing a numeric hash
Designing more useful rounding
Implementing other special methods
Optimization with the in-place operators
Summary
Design considerations and trade-offsLooking forward
8. Decorators and Mixins – Cross-cutting Aspects
Class and meaningConstructing the functionsConstructing the classSome class design principlesAspect-oriented programming
Using built-in decorators
Using standard library decorators
Using standard library mixin classes
Using the context manager mixin classTurning off a class feature
Writing a simple function decorator
Creating separate loggers
Parameterizing a decorator
Creating a method function decorator
Creating a class decorator
Adding method functions to a class
Using decorators for security
Summary
Design considerations and trade-offsLooking forward
2. Persistence and Serialization
Persistence and Serialization
9. Serializing and Saving – JSON, YAML, Pickle, CSV, and XML
Understanding persistence, class, state, and representationCommon Python terminologies
Filesystem and network considerations
Defining classes to support persistence
Rendering a blog and posts
Dumping and loading with JSON
Supporting JSON in our classesCustomizing JSON encodingCustomizing JSON decodingThe security and the eval() issueRefactoring the encode functionStandardizing the date stringWriting JSON to a file
Dumping and loading with YAML
Formatting YAML data on a fileExtending the YAML representationSecurity and safe loading
Dumping and loading with pickle
Designing a class for reliable pickle processingSecurity and the global issue
Dumping and loading with CSV
Dumping simple sequences to CSVLoading simple sequences from CSVHandling containers and complex classesDumping and loading multiple row types in a CSV fileFiltering CSV rows with an iteratorDumping and loading joined rows in a CSV file
Dumping and loading with XML
Dumping objects using string templatesDumping objects with xml.etree.ElementTreeLoading XML documents
Summary
Design considerations and trade-offsSchema evolutionLooking forward
10. Storing and Retrieving Objects via Shelve
Analyzing persistent object use casesThe ACID properties
Creating a shelf
Designing shelvable objects
Designing keys for our objectsGenerating surrogate keys for objectsDesigning a class with a simple keyDesigning classes for containers or collectionsReferring to objects via foreign keysDesigning CRUD operations for complex objects
Searching, scanning, and querying
Designing an access layer for shelve
Writing a demonstration script
Creating indexes to improve efficiency
Creating top-level indices
Adding yet more index maintenance
The writeback alternative to index updates
Schema evolution
Summary
Design considerations and trade-offsApplication software layersLooking forward
11. Storing and Retrieving Objects via SQLite
SQL databases, persistence, and objectsThe SQL data model – rows and tablesCRUD processing via SQL DML statementsQuerying rows with the SQL SELECT statementSQL transactions and the ACID propertiesDesigning primary and foreign database keys
Processing application data with SQL
Implementing class-like processing in pure SQL
Mapping Python objects to SQLite BLOB columns
Mapping Python objects to database rows manually
Designing an access layer for SQLiteImplementing container relationships
Improving performance with indices
Adding an ORM layer
Designing ORM-friendly classesBuilding the schema with the ORM layerManipulating objects with the ORM layer
Querying post objects given a tag string
Improving performance with indices
Schema evolution
Summary
Design considerations and trade-offsMapping alternativesKeys and key designsApplication software layersLooking forward
12. Transmitting and Sharing Objects
Class, state, and representation
Using HTTP and REST to transmit objects
Implementing CRUD operations via RESTImplementing non-CRUD operationsThe REST protocol and ACIDChoosing a representation – JSON, XML, or YAML
Implementing a REST server – WSGI and mod_wsgi
Creating a simple REST application and serverImplementing a REST clientDemonstrating and unit testing the RESTful services
Using Callable classes for WSGI applications
Designing RESTful object identifiersMultiple layers of REST servicesCreating the roulette serverCreating the roulette client
Creating a secure REST service
The WSGI Authentication application
Implementing REST with a web application framework
Using a message queue to transmit objects
Defining processesBuilding queues and supplying data
Summary
Design considerations and trade-offsSchema evolutionApplication software layersLooking forward
13. Configuration Files and Persistence
Configuration file use cases
Representation, persistence, state, and usability
Application configuration design patternsConfiguring via object constructionImplementing a configuration hierarchy
Storing the configuration in the INI files
Handling more literals via the eval() variants
Storing the configuration in PY files
Configuration via class definitionsConfiguration via SimpleNamespaceUsing Python with exec() for the configuration
Why is exec() a nonproblem?
Using ChainMap for defaults and overrides
Storing the configuration in JSON or YAML files
Using flattened JSON configurationsLoading a YAML configuration
Storing the configuration in property files
Parsing a properties fileUsing a properties file
Storing the configuration in XML files – PLIST and others
Customized XML configuration files
Summary
Design considerations and trade-offsCreating a shared configurationSchema evolutionLooking Forward
3. Testing, Debugging, Deploying, and Maintaining
Testing, Debugging, Deploying, and Maintaining
14. The Logging and Warning Modules
Creating a basic logCreating a shared class-level loggerConfiguring the loggersStarting up and shutting down the logging systemNaming the loggersExtending the logger levelsDefining handlers for multiple destinationsManaging the propagation rules
Configuration gotcha
Specializing logging for control, debug, audit, and security
Creating a debugging logCreating audit and security logs
Using the warnings module
Showing API changes with a warningShowing configuration problems with a warningShowing possible software problems with a warning
Advanced logging – the last few messages and network destinations
Building an automatic tail bufferSending logging messages to a remote processPreventing queue overrun
Summary
Design considerations and trade-offsLooking forward
15. Designing for Testability
Defining and isolating units for testingMinimizing the dependenciesCreating simple unit testsCreating a test suiteIncluding edge and corner casesMocking dependencies for testingUsing more mocks to test more behaviors
Using doctest to define test cases
Combining doctest and unittestCreating a more complete test package
Using setup and teardown
Using setup and teardown with OS resourcesUsing setup and teardown with databases
The TestCase class hierarchy
Using externally defined expected results
Automated integration or performance testing
Summary
Design considerations and trade-offsLooking forward
16. Coping With the Command Line
The OS interface and the command lineArguments and options
Parsing the command line with argparse
A simple on/off optionAn option with an argumentPositional argumentsAll other arguments--version display and exit--help display and exit
Integrating command-line options and environment variables
Providing more configurable defaultsOverriding configuration file settings with environment variablesOverriding environment variables with the configuration filesMaking the configuration aware of the None values
Customizing the help output
Creating a top-level main() function
Ensuring DRY for the configurationManaging nested configuration contexts
Programming In The Large
Designing command classesAdding the analysis command subclassAdding and packaging more features into an applicationDesigning a higher-level composite command
Additional composite command design patterns
Integrating with other applications
Summary
Design considerations and trade-offsLooking forward
17. The Module and Package Design
Designing a moduleSome module design patternsModule versus classThe expected content of a module
Whole module versus module items
Designing a package
Designing a module-package hybridDesigning a package with alternate implementations
Designing a main script and the __main__ module
Creating an executable script fileCreating a __main__ moduleProgramming in the large
Designing long-running applications
Organizing code into src, bin, and test
Installing Python modules
Summary
Design considerations and trade-offsLooking forward
18. Quality and Documentation
Writing docstrings for the help() function
Using pydoc for documentation
Better output via the RST markup
Blocks of textThe RST inline markupRST directivesLearning RST
Writing effective docstrings
Writing file-level docstrings, including modules and packages
Writing API details in RST markupWriting class and method function docstringsWriting function docstrings
More sophisticated markup techniques
Using Sphinx to produce the documentation
Using the Sphinx quickstartWriting the Sphinx documentationFilling in the 4+1 views for documentationWriting the implementation documentCreating the Sphinx cross-referencesRefactoring Sphinx files into directories
Writing the documentation
Literate programming
Use cases for literate programmingWorking with a literate programming tool
Summary
Design considerations and trade-offs
Index

Content preview from Mastering Object-oriented Python

Dumping and loading with pickle

The pickle module is Python's native format to make objects persistent.

The Python Standard Library says this about pickle:

The pickle module can transform a complex object into a byte stream and it can transform the byte stream into an object with the same internal structure. Perhaps the most obvious thing to do with these byte streams is to write them onto a file, but it is also conceivable to send them across a network or store them in a database.

The focus of pickle is Python and only Python. This is not a data interchange format such as JSON, YAML, CSV, or XML that can be used with applications written in other languages.

The pickle module is tightly integrated with Python in a variety of ways. For example, the ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Mastering Object-Oriented Python - Second Edition

Publisher Resources

ISBN: 9781783280971

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Mastering Object-oriented Python

by Steven F. Lott

Dumping and loading with pickle

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

More than 5,000 organizations count on O’Reilly

Julian F.

Addison B.

Amir M.

Mark W.

You might also like

Mastering Object-Oriented Python - Second Edition

Object-Oriented Python

Python Object-Oriented Programming - Fourth Edition

Object-oriented Programming with Python - A Practical Guide

Publisher Resources