Expert Python Programming - Fourth Edition

Book description

Gain a deep understanding of building, maintaining, packaging, and shipping robust Python applications

Key Features

  • Discover the new features of Python, such as dictionary merge, the zoneinfo module, and structural pattern matching
  • Create manageable code to run in various environments with different sets of dependencies
  • Implement effective Python data structures and algorithms to write, test, and optimize code

Book Description

This new edition of Expert Python Programming provides you with a thorough understanding of the process of building and maintaining Python apps. Complete with best practices, useful tools, and standards implemented by professional Python developers, this fourth edition has been extensively updated. Throughout this book, you'll get acquainted with the latest Python improvements, syntax elements, and interesting tools to boost your development efficiency.

The initial few chapters will allow experienced programmers coming from different languages to transition to the Python ecosystem. You will explore common software design patterns and various programming methodologies, such as event-driven programming, concurrency, and metaprogramming. You will also go through complex code examples and try to solve meaningful problems by bridging Python with C and C++, writing extensions that benefit from the strengths of multiple languages. Finally, you will understand the complete lifetime of any application after it goes live, including packaging and testing automation.

By the end of this book, you will have gained actionable Python programming insights that will help you effectively solve challenging problems.

What you will learn

  • Explore modern ways of setting up repeatable and consistent Python development environments
  • Effectively package Python code for community and production use
  • Learn modern syntax elements of Python programming, such as f-strings, enums, and lambda functions
  • Demystify metaprogramming in Python with metaclasses
  • Write concurrent code in Python
  • Extend and integrate Python with code written in C and C++

Who this book is for

The Python programming book is intended for expert programmers who want to learn Python's advanced-level concepts and latest features.

Anyone who has basic Python skills should be able to follow the content of the book, although it might require some additional effort from less experienced programmers. It should also be a good introduction to Python 3.9 for those who are still a bit behind and continue to use other older versions.

Table of contents

  1. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
    4. Get in touch
  2. Current Status of Python
    1. Where are we now and where are we going?
    2. What to do with Python 2
    3. Keeping up to date
      1. PEP documents
      2. Active communities
      3. Other resources
    4. Summary
  3. Modern Python Development Environments
    1. Technical requirements
    2. Python's packaging ecosystem
      1. Installing Python packages using pip
    3. Isolating the runtime environment
      1. Application-level isolation versus system-level isolation
    4. Application-level environment isolation
      1. Poetry as a dependency management system
    5. System-level environment isolation
      1. Containerization versus virtualization
      2. Virtual environments using Docker
        1. Writing your first Dockerfile
        2. Running containers
        3. Setting up complex environments
        4. Useful Docker and Docker Compose recipes for Python
      3. Virtual development environments using Vagrant
    6. Popular productivity tools
      1. Custom Python shells
      2. Using IPython
      3. Incorporating shells in your own scripts and programs
      4. Interactive debuggers
      5. Other productivity tools
    7. Summary
  4. New Things in Python
    1. Technical requirements
    2. Recent language additions
      1. Dictionary merge and update operators
        1. Alternative – Dictionary unpacking
        2. Alternative – ChainMap from the collections module
      2. Assignment expressions
      3. Type-hinting generics
      4. Positional-only parameters
      5. zoneinfo module
      6. graphlib module
    3. Not that new, but still shiny
      1. breakpoint() function
      2. Development mode
      3. Module-level __getattr__() and __dir__() functions
      4. Formatting strings with f-strings
      5. Underscores in numeric literals
      6. secrets module
    4. What may come in the future?
      1. Union types with the | operator
      2. Structural pattern matching
    5. Summary
  5. Python in Comparison with Other Languages
    1. Technical requirements
    2. Class model and object-oriented programming
      1. Accessing super-classes
      2. Multiple inheritance and Method Resolution Order
      3. Class instance initialization
      4. Attribute access patterns
      5. Descriptors
        1. Real-life example – lazily evaluated attributes
      6. Properties
    3. Dynamic polymorphism
      1. Operator overloading
        1. Dunder methods (language protocols)
        2. Comparison to C++
      2. Function and method overloading
        1. Single-dispatch functions
    4. Data classes
    5. Functional programming
      1. Lambda functions
      2. The map(), filter(), and reduce() functions
      3. Partial objects and partial functions
      4. Generators
      5. Generator expressions
      6. Decorators
    6. Enumerations
    7. Summary
  6. Interfaces, Patterns, and Modularity
    1. Technical requirements
    2. Interfaces
      1. A bit of history: zope.interface
      2. Using function annotations and abstract base classes
        1. Using collections.abc
      3. Interfaces through type annotations
    3. Inversion of control and dependency injection
      1. Inversion of control in applications
      2. Using dependency injection frameworks
    4. Summary
  7. Concurrency
    1. Technical requirements
    2. What is concurrency?
    3. Multithreading
      1. What is multithreading?
      2. How Python deals with threads
      3. When should we use multithreading?
        1. Application responsiveness
        2. Multiuser applications
        3. Work delegation and background processing
      4. An example of a multithreaded application
        1. Using one thread per item
        2. Using a thread pool
        3. Using two-way queues
        4. Dealing with errors in threads
        5. Throttling
    4. Multiprocessing
      1. The built-in multiprocessing module
      2. Using process pools
      3. Using multiprocessing.dummy as the multithreading interface
    5. Asynchronous programming
      1. Cooperative multitasking and asynchronous I/O
      2. Python async and await keywords
      3. A practical example of asynchronous programming
      4. Integrating non-asynchronous code with async using futures
        1. Executors and futures
        2. Using executors in an event loop
    6. Summary
  8. Event-Driven Programming
    1. Technical requirements
    2. What exactly is event-driven programming?
      1. Event-driven != asynchronous
      2. Event-driven programming in GUIs
      3. Event-driven communication
    3. Various styles of event-driven programming
      1. Callback-based style
      2. Subject-based style
      3. Topic-based style
    4. Event-driven architectures
      1. Event and message queues
    5. Summary
  9. Elements of Metaprogramming
    1. Technical requirements
    2. What is metaprogramming?
    3. Using decorators to modify function behavior before use
      1. One step deeper: class decorators
    4. Intercepting the class instance creation process
    5. Metaclasses
      1. The general syntax
      2. Metaclass usage
      3. Metaclass pitfalls
      4. Using the __init__subclass__() method as an alternative to metaclasses
    6. Code generation
      1. exec, eval, and compile
      2. The abstract syntax tree
      3. Import hooks
      4. Notable examples of code generation in Python
        1. Falcon's compiled router
        2. Hy
    7. Summary
  10. Bridging Python with C and C++
    1. Technical requirements
    2. C and C++ as the core of Python extensibility
    3. Compiling and loading Python C extensions
    4. The need to use extensions
      1. Improving performance in critical code sections
      2. Integrating existing code written in different languages
      3. Integrating third-party dynamic libraries
      4. Creating efficient custom datatypes
    5. Writing extensions
      1. Pure C extensions
        1. A closer look at the Python/C API
        2. Calling and binding conventions
        3. Exception handling
        4. Releasing GIL
        5. Reference counting
      2. Writing extensions with Cython
        1. Cython as a source-to-source compiler
        2. Cython as a language
    6. Downsides of using extensions
      1. Additional complexity
      2. Harder debugging
    7. Interfacing with dynamic libraries without extensions
      1. The ctypes module
        1. Loading libraries
        2. Calling C functions using ctypes
        3. Passing Python functions as C callbacks
      2. CFFI
    8. Summary
  11. Testing and Quality Automation
    1. Technical requirements
    2. The principles of test-driven development
    3. Writing tests with pytest
      1. Test parameterization
      2. pytest's fixtures
      3. Using fakes
      4. Mocks and the unittest.mock module
    4. Quality automation
      1. Test coverage
      2. Style fixers and code linters
      3. Static type analysis
    5. Mutation testing
    6. Useful testing utilities
      1. Faking realistic data values
      2. Faking time values
    7. Summary
  12. Packaging and Distributing Python Code
    1. Technical requirements
    2. Packaging and distributing libraries
      1. The anatomy of a Python package
        1. setup.py
        2. setup.cfg
        3. MANIFEST.in
        4. Essential package metadata
        5. Trove classifiers
      2. Types of package distributions
        1. sdist distributions
        2. bdist and wheel distributions
      3. Registering and publishing packages
      4. Package versioning and dependency management
        1. The SemVer standard for semantic versioning
        2. CalVer for calendar versioning
      5. Installing your own packages
        1. Installing packages directly from sources
        2. Installing packages in editable mode
      6. Namespace packages
      7. Package scripts and entry points
    3. Packaging applications and services for the web
      1. The Twelve-Factor App manifesto
      2. Leveraging Docker
      3. Handling environment variables
      4. The role of environment variables in application frameworks
    4. Creating standalone executables
      1. When standalone executables are useful
      2. Popular tools
        1. PyInstaller
        2. cx_Freeze
        3. py2exe and py2app
      3. Security of Python code in executable packages
    5. Summary
  13. Observing Application Behavior and Performance
    1. Technical requirements
    2. Capturing errors and logs
      1. Python logging essentials
        1. Logging system components
        2. Logging configuration
      2. Good logging practices
      3. Distributed logging
      4. Capturing errors for later review
    3. Instrumenting code with custom metrics
      1. Using Prometheus
    4. Distributed application tracing
      1. Distributed tracing with Jaeger
    5. Summary
  14. Code Optimization
    1. Technical requirements
    2. Common culprits for bad performance
      1. Code complexity
        1. Cyclomatic complexity
        2. The big O notation
      2. Excessive resource allocation and leaks
      3. Excessive I/O and blocking operations
    3. Code profiling
      1. Profiling CPU usage
        1. Macro-profiling
        2. Micro-profiling
      2. Profiling memory usage
        1. Using the objgraph module
        2. C code memory leaks
    4. Reducing complexity by choosing appropriate data structures
      1. Searching in a list
      2. Using sets
      3. Using the collections module
        1. deque
        2. defaultdict
        3. namedtuple
    5. Leveraging architectural trade-offs
      1. Using heuristics and approximation algorithms
      2. Using task queues and delayed processing
      3. Using probabilistic data structures
      4. Caching
        1. Deterministic caching
        2. Non-deterministic caching
    6. Summary
  15. Other Books You May Enjoy
  16. Index

Product information

  • Title: Expert Python Programming - Fourth Edition
  • Author(s): Michal Jaworski, Tarek Ziadé
  • Release date: May 2021
  • Publisher(s): Packt Publishing
  • ISBN: 9781801071109