Fluent Python, 2nd Edition

Book description

Python’s simplicity lets you become productive quickly, but often this means you aren’t using everything it has to offer. With the updated edition of this hands-on guide, you’ll learn how to write effective, modern Python 3 code by leveraging its best ideas.

Don’t waste time bending Python to fit patterns you learned in other languages. Discover and apply idiomatic Python 3 features beyond your past experience. Author Luciano Ramalho guides you through Python’s core language features and libraries and teaches you how to make your code shorter, faster, and more readable.

Featuring major updates throughout the book, Fluent Python, second edition, covers:

  • Special methods: The key to the consistent behavior of Python objects
  • Data structures: Sequences, dicts, sets, Unicode, and data classes
  • Functions as objects: First-class functions, related design patterns, and type hints in function declarations
  • Object-oriented idioms: Composition, inheritance, mixins, interfaces, operator overloading, static typing and protocols
  • Control flow: Context managers, generators, coroutines, async/await, and thread/process pools
  • Metaprogramming: Properties, attribute descriptors, class decorators, and new class metaprogramming hooks that are simpler than metaclasses

Publisher resources

View/Submit Errata

Table of contents

  1. Preface
    1. Who This Book Is For
    2. Who This Book Is Not For
    3. How This Book Is Organized
    4. Hands-On Approach
    5. Hardware Used for Timings
    6. Soapbox: My Personal Perspective
    7. Python Jargon
    8. Python Version Covered
    9. Conventions Used in This Book
    10. Using Code Examples
    11. How to Contact Us
    12. Acknowledgments
  2. I. Prologue
  3. 1. The Python Data Model
    1. What’s new in this chapter
    2. A Pythonic Card Deck
    3. How Special Methods Are Used
      1. Emulating Numeric Types
      2. String Representation
      3. Arithmetic Operators
      4. Boolean Value of a Custom Type
      5. Collection API
    4. Overview of Special Methods
    5. Why len Is Not a Method
    6. Chapter Summary
    7. Further Reading
  4. II. Data Structures
  5. 2. An Array of Sequences
    1. What’s new in this chapter
    2. Overview of Built-In Sequences
    3. List Comprehensions and Generator Expressions
      1. List Comprehensions and Readability
      2. Listcomps Versus map and filter
      3. Cartesian Products
      4. Generator Expressions
    4. Tuples Are Not Just Immutable Lists
      1. Tuples as Records
      2. Unpacking
      3. Nested Tuple Unpacking
      4. Tuples as Immutable Lists
      5. Tuple versus list methods
    5. Slicing
      1. Why Slices and Range Exclude the Last Item
      2. Slice Objects
      3. Multidimensional Slicing and Ellipsis
      4. Assigning to Slices
    6. Using + and * with Sequences
      1. Building Lists of Lists
    7. Augmented Assignment with Sequences
      1. A += Assignment Puzzler
    8. list.sort and the sorted Built-In Function
    9. Managing Ordered Sequences with bisect
      1. Searching with bisect
      2. Inserting with bisect.insort
    10. When a List Is Not the Answer
      1. Arrays
      2. Memory Views
      3. NumPy
      4. Deques and Other Queues
    11. Chapter Summary
    12. Further Reading
  6. 3. Dictionaries and Sets
    1. What’s new in this chapter
    2. Standard API of Mapping Types
    3. dict Comprehensions
    4. Overview of Common Mapping Methods
      1. Handling Missing Keys with setdefault
    5. Mappings with Flexible Key Lookup
      1. defaultdict: Another Take on Missing Keys
      2. The __missing__ Method
    6. Variations of dict
    7. Building custom mappings
      1. Subclassing UserDict
    8. Immutable Mappings
    9. Dictionary views
    10. Set Theory
      1. Set Literals
      2. Set Comprehensions
      3. Set Operations
      4. Set operations on dict views
    11. Internals of sets and dicts
      1. A Performance Experiment
      2. Set hash tables under the hood
      3. The hash table algorithm
      4. Hash table usage in dict
      5. Key-sharing dictionary
      6. Practical Consequences of How dict Works
    12. Chapter Summary
    13. Further Reading
  7. 4. Text versus Bytes
    1. What’s new in this chapter
    2. Character Issues
    3. Byte Essentials
    4. Basic Encoders/Decoders
    5. Understanding Encode/Decode Problems
      1. Coping with UnicodeEncodeError
      2. Coping with UnicodeDecodeError
      3. SyntaxError When Loading Modules with Unexpected Encoding
      4. How to Discover the Encoding of a Byte Sequence
      5. BOM: A Useful Gremlin
    6. Handling Text Files
      1. Beware of Encoding Defaults
    7. Normalizing Unicode for Reliable Comparisons
      1. Case Folding
      2. Utility Functions for Normalized Text Matching
      3. Extreme “Normalization”: Taking Out Diacritics
    8. Sorting Unicode Text
      1. Sorting with the Unicode Collation Algorithm
    9. The Unicode Database
      1. Finding characters by name
      2. Numeric meaning of characters
    10. Dual-Mode str and bytes APIs
      1. str Versus bytes in Regular Expressions
      2. str Versus bytes in os Functions
    11. Multi-character emojis
      1. Country flags
      2. Skin tones
      3. Rainbow flag and other ZWJ sequences
    12. Chapter Summary
    13. Further Reading
  8. 5. Record-like data structures
    1. What’s new in this chapter
    2. Overview of data class builders
      1. Main features
    3. Classic Named Tuples
    4. Typed Named Tuples
    5. Type hints 101
      1. No runtime effect
      2. Variable annotation Syntax
      3. The meaning of variable annotations
    6. More about @dataclass
      1. Field options
      2. Post-init processing
      3. Typed class attributes
      4. Initialization variables that are not fields
      5. @dataclass Example: Dublin Core Resource Record
    7. Data class as a code smell
      1. Data class as scaffolding
      2. Data class as intermediate representation
    8. Parsing binary records with struct
      1. Structs and Memory Views
      2. Should we use struct?
    9. Chapter Summary
    10. Further Reading
  9. 6. Object References, Mutability, and Recycling
    1. What’s new in this chapter
    2. Variables Are Not Boxes
    3. Identity, Equality, and Aliases
      1. Choosing Between == and is
      2. The Relative Immutability of Tuples
    4. Copies Are Shallow by Default
      1. Deep and Shallow Copies of Arbitrary Objects
    5. Function Parameters as References
      1. Mutable Types as Parameter Defaults: Bad Idea
      2. Defensive Programming with Mutable Parameters
    6. del and Garbage Collection
    7. Weak References
      1. The WeakValueDictionary Skit
      2. Limitations of Weak References
    8. Tricks Python Plays with Immutables
    9. Chapter Summary
    10. Further Reading
  10. III. Functions as Objects
  11. 7. First-Class Functions
    1. What’s new in this chapter
    2. Treating a Function Like an Object
    3. Higher-Order Functions
      1. Modern Replacements for map, filter, and reduce
    4. Anonymous Functions
    5. The Nine Flavors of Callable Objects
    6. User-Defined Callable Types
    7. Function Introspection
    8. From Positional to Keyword-Only Parameters
      1. Positional-only parameters
    9. Retrieving Information About Parameters
    10. Packages for Functional Programming
      1. The operator Module
      2. Freezing Arguments with functools.partial
    11. Chapter Summary
    12. Further Reading
  12. 8. Type Hints in Functions
    1. What’s new in this chapter
    2. About gradual typing
    3. Gradual typing in practice
      1. Starting with Mypy
      2. Making Mypy More Strict
      3. A Default Parameter Value
      4. Using None as a default
      5. Type hints for Python 2.7 and 3.x
    4. Types are defined by supported operations
    5. Types usable in annotations
      1. The Any type
      2. Simple types and classes
      3. Optional and Union types
      4. Generic collections
      5. Tuple
      6. Generic mappings
      7. TypedDict
      8. Abstract Base Classes
      9. Iterable
      10. Parameterized generics and TypeVar
      11. Protocols
      12. Callable
      13. NoReturn
    6. Annotating positional-only and variadic parameters
    7. Chapter summary
    8. Further Reading
  13. 9. Decorators and Closures
    1. What’s new in this chapter
    2. Decorators 101
    3. When Python Executes Decorators
    4. Registration decorators
    5. Variable Scope Rules
    6. Closures
    7. The nonlocal Declaration
    8. Implementing a Simple Decorator
      1. How It Works
    9. Decorators in the Standard Library
      1. Memoization with functools.cache
      2. Using lru_cache
      3. Single Dispatch Generic Functions
    10. Parameterized Decorators
      1. A Parameterized Registration Decorator
      2. The Parameterized Clock Decorator
      3. A class-based clock decorator
    11. Chapter Summary
    12. Further Reading
  14. 10. Design Patterns with First-Class Functions
    1. What’s new in this chapter
    2. Case Study: Refactoring Strategy
      1. Classic Strategy
      2. Function-Oriented Strategy
      3. Choosing the Best Strategy: Simple Approach
      4. Finding Strategies in a Module
    3. Decorator-Enhanced Strategy Pattern
    4. Command
    5. Chapter Summary
    6. Further Reading
  15. IV. Classes and Protocols
  16. 11. A Pythonic Object
    1. What’s new in this chapter
    2. Object Representations
    3. Vector Class Redux
    4. An Alternative Constructor
    5. classmethod Versus staticmethod
    6. Formatted Displays
    7. A Hashable Vector2d
    8. Private and “Protected” Attributes in Python
    9. Saving Memory with __slots__
      1. The Problems with __slots__
    10. Overriding Class Attributes
    11. Chapter Summary
    12. Further Reading
  17. 12. Sequence Hacking, Hashing, and Slicing
    1. What’s new in this chapter
    2. Vector: A User-Defined Sequence Type
    3. Vector Take #1: Vector2d Compatible
    4. Protocols and Duck Typing
    5. Vector Take #2: A Sliceable Sequence
      1. How Slicing Works
      2. A Slice-Aware __getitem__
    6. Vector Take #3: Dynamic Attribute Access
    7. Vector Take #4: Hashing and a Faster ==
    8. Vector Take #5: Formatting
    9. Chapter Summary
    10. Further Reading
  18. 13. Interfaces, Protocols, and ABCs
    1. The Typing Map
    2. What’s new in this chapter
    3. Two kinds of protocols
    4. Programming ducks
      1. Python Digs Sequences
      2. Monkey-Patching: Implementing a Protocol at Runtime
      3. Defensive programming and “fail fast”
    5. Goose typing
      1. Subclassing an ABC
      2. ABCs in the Standard Library
      3. Defining and Using an ABC
      4. ABC Syntax Details
      5. Subclassing the Tombola ABC
      6. A Virtual Subclass of Tombola
      7. Usage of register in Practice
      8. Structural typing with ABCs
    6. Static protocols
      1. The typed double function
      2. Runtime checkable static protocols
      3. Supporting a static protocol
      4. Designing a static protocol
      5. Extending a protocol
      6. The numbers ABCs and numeric protocols
    7. Chapter Summary
    8. Further Reading
  19. 14. Inheritance: For Good or For Worse
    1. What’s new in this chapter
    2. Subclassing Built-In Types Is Tricky
    3. Multiple Inheritance and Method Resolution Order
    4. Multiple Inheritance in the Real World
    5. Coping with Multiple Inheritance
      1. 1. Distinguish Interface Inheritance from Implementation Inheritance
      2. 2. Make Interfaces Explicit with ABCs
      3. 3. Use Mixins for Code Reuse
      4. 4. Make Mixins Explicit by Naming
      5. 5. An ABC May Also Be a Mixin; The Reverse Is Not True
      6. 6. Don’t Subclass from More Than One Concrete Class
      7. 7. Provide Aggregate Classes to Users
      8. 8. “Favor Object Composition Over Class Inheritance.”
      9. Tkinter: The Good, the Bad, and the Ugly
    6. A Modern Example: Mixins in Django Generic Views
    7. Chapter Summary
    8. Further Reading
  20. 15. More About Type Hints
  21. 16. Operator Overloading: Doing It Right
    1. What’s new in this chapter
    2. Operator Overloading 101
    3. Unary Operators
    4. Overloading + for Vector Addition
    5. Overloading * for Scalar Multiplication
    6. Using @ as an infix operator
    7. Wrapping-up arithmetic operators
    8. Rich Comparison Operators
    9. Augmented Assignment Operators
    10. Chapter Summary
    11. Further Reading
  22. 17. Iterables, Iterators, and Generators
    1. What’s new in this chapter
    2. A Sequence of Words
    3. Why Sequences Are Iterable: The iter Function
    4. Iterables Versus Iterators
    5. Sentence classes with __iter__
      1. Sentence Take #2: A Classic Iterator
      2. Don’t make the iterable an iterator for itself
      3. Sentence Take #3: A Generator Function
      4. How a Generator Works
    6. Lazy sentences
      1. Sentence Take #4: Lazy Generator
      2. Sentence Take #5: Lazy Generator Expression
    7. Generator Expressions: When to Use Them
    8. Another Example: Arithmetic Progression Generator
      1. Arithmetic Progression with itertools
    9. Generator Functions in the Standard Library
    10. Subgenerators with yield from
      1. Reinventing chain.
      2. Traversing a tree
    11. Iterable Reducing Functions
    12. A Closer Look at the iter Function
    13. Case Study: Generators in a Database Conversion Utility
    14. Generators as Coroutines
    15. Chapter Summary
    16. Further Reading
  23. 18. Context Managers and else Blocks
    1. What’s new in this chapter
    2. Do This, Then That: else Blocks Beyond if
    3. Context Managers and with Blocks
    4. The contextlib Utilities
    5. Using @contextmanager
    6. Chapter Summary
    7. Further Reading
  24. 19. Classic Coroutines
    1. What’s new in this chapter
    2. How Coroutines Evolved from Generators
    3. Basic Behavior of a Generator Used as a Coroutine
    4. Example: Coroutine to Compute a Running Average
    5. Decorators for Coroutine Priming
    6. Coroutine Termination and Exception Handling
    7. Returning a Value from a Coroutine
    8. Using yield from
      1. Pipelines of coroutines
    9. The Meaning of yield from
      1. Basic behavior of yield from
      2. Exception handling in yield from
    10. Use Case: Coroutines for Discrete Event Simulation
      1. About Discrete Event Simulations
      2. The Taxi Fleet Simulation
    11. Chapter Summary
    12. Further Reading
  25. 20. Concurrency Models in Python
    1. What’s new in this chapter
    2. A Bit of Jargon
      1. Processes, threads, and Python’s Infamous GIL
    3. A Concurrent Hello World
      1. Spinner with threading
      2. Spinner with multiprocessing
      3. Spinner with asyncio
      4. Supervisors Side-by-side
    4. The Real Impact of the GIL
      1. Quick Quiz
    5. A Homegrown Process Pool
      1. Process-based Solution
      2. Understanding the Elapsed Times
      3. Code for the Multi-core Prime Checker
      4. Thread-based Non-solution
    6. The Big Picture
      1. System Administration
      2. Data Science
      3. Server-side Web/Mobile Development
      4. WSGI Application servers
      5. Distributed task queues
    7. Chapter Summary
    8. Further Reading
      1. Concurrency with threads and processes
      2. The GIL
      3. Concurrency beyond the standard library
      4. Concurrency and scalability beyond Python
  26. 21. Concurrency with Futures
    1. What’s new in this chapter
    2. Concurrent Web Downloads
      1. A Sequential Download Script
      2. Downloading with concurrent.futures
      3. Where Are the Futures?
    3. Launching Processes with concurrent.futures
      1. Multi-core Prime Checker Redux
    4. Experimenting with Executor.map
    5. Downloads with Progress Display and Error Handling
      1. Error Handling in the flags2 Examples
      2. Using futures.as_completed
    6. Chapter Summary
    7. Further Reading
  27. 22. Asynchronous Programming
    1. What’s New in this Chapter
    2. A few definitions
    3. Example: Probing Domains
      1. Guido’s trick to read asynchronous code
    4. New concept: awaitable
    5. Downloading with asyncio and aiohttp
      1. The Secret of Native Coroutines: Humble Generators
      2. The all-or-nothing problem
    6. Asynchronous Context Managers
    7. Enhancing the asyncio downloader
      1. Using asyncio.as_completed and a semaphore
      2. Using an Executor to Avoid Blocking the Event Loop
      3. Making Multiple Requests for Each Download
    8. Writing asyncio Servers
      1. A FastAPI Web Service
      2. An asyncio TCP Server
    9. Asynchronous iteration and asynchronous iterables
      1. Asynchronous Generator Functions
      2. Async Comprehensions and Async Generator Expressions
    10. Async beyond asyncio: Curio
    11. How Async Works and How It Doesn’t
      1. Running Circles Around Blocking Calls
      2. The Myth of I/O Bound Systems
      3. Avoiding CPU-bound Traps
    12. Chapter Summary
    13. Further Reading
  28. 23. Dynamic Attributes and Properties
    1. What’s new in this chapter
    2. Data Wrangling with Dynamic Attributes
      1. Exploring JSON-Like Data with Dynamic Attributes
      2. The Invalid Attribute Name Problem
      3. Flexible Object Creation with __new__
    3. Computed Properties
      1. Step 1: Data-driven Attribute Creation
      2. Step 2: Property to Retrieve a Linked Record
      3. Step 3: Property Overriding an Existing Attribute
      4. Step 4: Bespoke Property Cache
      5. Step 5: Caching Properties with functools
    4. Using a Property for Attribute Validation
      1. LineItem Take #1: Class for an Item in an Order
      2. LineItem Take #2: A Validating Property
    5. A Proper Look at Properties
      1. Properties Override Instance Attributes
      2. Property Documentation
    6. Coding a Property Factory
    7. Handling Attribute Deletion
    8. Essential Attributes and Functions for Attribute Handling
      1. Special Attributes that Affect Attribute Handling
      2. Built-In Functions for Attribute Handling
      3. Special Methods for Attribute Handling
    9. Chapter Summary
    10. Further Reading
  29. 24. Attribute Descriptors
    1. What’s new in this chapter
    2. Descriptor Example: Attribute Validation
      1. LineItem Take #3: A Simple Descriptor
      2. LineItem Take #4: Automatic Storage Attribute Names
      3. LineItem Take #5: A New Descriptor Type
    3. Overriding Versus Non-Overriding Descriptors
      1. Overriding Descriptors
      2. Overriding Descriptor Without __get__
      3. Non-overriding Descriptor
      4. Overwriting a Descriptor in the Class
    4. Methods Are Descriptors
    5. Descriptor Usage Tips
    6. Descriptor docstring and Overriding Deletion
    7. Chapter Summary
    8. Further Reading

Product information

  • Title: Fluent Python, 2nd Edition
  • Author(s): Luciano Ramalho
  • Release date: November 2021
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781492056355