Fluent Python, 2nd Edition

Fluent Python, 2nd Edition

by Luciano Ramalho
Released April 2022
Publisher(s): O'Reilly Media, Inc.
ISBN: 9781492056355

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon Buy on ebooks.com
Start your free trial

Book description

Don't waste time bending Python to fit patterns you've learned in other languages. Python's simplicity lets you become productive quickly, but often this means you aren't using everything the language has to offer. With the updated edition of this hands-on guide, you'll learn how to write effective, modern Python 3 code by leveraging its best ideas.

Discover and apply idiomatic Python 3 features beyond your past experience. Author Luciano Ramalho guides you through Python's core language features and libraries and teaches you how to make your code shorter, faster, and more readable.

Complete with major updates throughout, this new edition features five parts that work as five short books within the book:

  • Data structures: Sequences, dicts, sets, Unicode, and data classes
  • Functions as objects: First-class functions, related design patterns, and type hints in function declarations
  • Object-oriented idioms: Composition, inheritance, mixins, interfaces, operator overloading, protocols, and more static types
  • Control flow: Context managers, generators, coroutines, async/await, and thread/process pools
  • Metaprogramming: Properties, attribute descriptors, class decorators, and new class metaprogramming hooks that replace or simplify metaclasses

Table of contents

  1. Preface
    1. Who This Book Is For
    2. Who This Book Is Not For
    3. Five Books in One
      1. How the Book Is Organized
    4. Hands-On Approach
    5. Soapbox: My Personal Perspective
    6. Companion Website: fluentpython.com
    7. Conventions Used in This Book
    8. Using Code Examples
    9. O’Reilly Online Learning
    10. How to Contact Us
    11. Acknowledgments
      1. Acknowledgments for the First Edition
  2. I. Data Structures
  3. 1. The Python Data Model
    1. What’s New in This Chapter
    2. A Pythonic Card Deck
    3. How Special Methods Are Used
      1. Emulating Numeric Types
      2. String Representation
      3. Boolean Value of a Custom Type
      4. Collection API
    4. Overview of Special Methods
    5. Why len Is Not a Method
    6. Chapter Summary
    7. Further Reading
  4. 2. An Array of Sequences
    1. What’s New in This Chapter
    2. Overview of Built-In Sequences
    3. List Comprehensions and Generator Expressions
      1. List Comprehensions and Readability
      2. Listcomps Versus map and filter
      3. Cartesian Products
      4. Generator Expressions
    4. Tuples Are Not Just Immutable Lists
      1. Tuples as Records
      2. Tuples as Immutable Lists
      3. Comparing Tuple and List Methods
    5. Unpacking Sequences and Iterables
      1. Using * to Grab Excess Items
      2. Unpacking with * in Function Calls and Sequence Literals
      3. Nested Unpacking
    6. Pattern Matching with Sequences
      1. Pattern Matching Sequences in an Interpreter
    7. Slicing
      1. Why Slices and Ranges Exclude the Last Item
      2. Slice Objects
      3. Multidimensional Slicing and Ellipsis
      4. Assigning to Slices
    8. Using + and * with Sequences
      1. Building Lists of Lists
      2. Augmented Assignment with Sequences
      3. A += Assignment Puzzler
    9. list.sort Versus the sorted Built-In
    10. When a List Is Not the Answer
      1. Arrays
      2. Memory Views
      3. NumPy
      4. Deques and Other Queues
    11. Chapter Summary
    12. Further Reading
  5. 3. Dictionaries and Sets
    1. What’s New in This Chapter
    2. Modern dict Syntax
      1. dict Comprehensions
      2. Unpacking Mappings
      3. Merging Mappings with |
    3. Pattern Matching with Mappings
    4. Standard API of Mapping Types
      1. What Is Hashable
      2. Overview of Common Mapping Methods
      3. Inserting or Updating Mutable Values
    5. Automatic Handling of Missing Keys
      1. defaultdict: Another Take on Missing Keys
      2. The __missing__ Method
      3. Inconsistent Usage of __missing__ in the Standard Library
    6. Variations of dict
      1. collections.OrderedDict
      2. collections.ChainMap
      3. collections.Counter
      4. shelve.Shelf
      5. Subclassing UserDict Instead of dict
    7. Immutable Mappings
    8. Dictionary Views
    9. Practical Consequences of How dict Works
    10. Set Theory
      1. Set Literals
      2. Set Comprehensions
    11. Practical Consequences of How Sets Work
      1. Set Operations
    12. Set Operations on dict Views
    13. Chapter Summary
    14. Further Reading
  6. 4. Unicode Text Versus Bytes
    1. What’s New in This Chapter
    2. Character Issues
    3. Byte Essentials
    4. Basic Encoders/Decoders
    5. Understanding Encode/Decode Problems
      1. Coping with UnicodeEncodeError
      2. Coping with UnicodeDecodeError
      3. SyntaxError When Loading Modules with Unexpected Encoding
      4. How to Discover the Encoding of a Byte Sequence
      5. BOM: A Useful Gremlin
    6. Handling Text Files
      1. Beware of Encoding Defaults
    7. Normalizing Unicode for Reliable Comparisons
      1. Case Folding
      2. Utility Functions for Normalized Text Matching
      3. Extreme “Normalization”: Taking Out Diacritics
    8. Sorting Unicode Text
      1. Sorting with the Unicode Collation Algorithm
    9. The Unicode Database
      1. Finding Characters by Name
      2. Numeric Meaning of Characters
    10. Dual-Mode str and bytes APIs
      1. str Versus bytes in Regular Expressions
      2. str Versus bytes in os Functions
    11. Chapter Summary
    12. Further Reading
  7. 5. Data Class Builders
    1. What’s New in This Chapter
    2. Overview of Data Class Builders
      1. Main Features
    3. Classic Named Tuples
    4. Typed Named Tuples
    5. Type Hints 101
      1. No Runtime Effect
      2. Variable Annotation Syntax
      3. The Meaning of Variable Annotations
    6. More About @dataclass
      1. Field Options
      2. Post-init Processing
      3. Typed Class Attributes
      4. Initialization Variables That Are Not Fields
      5. @dataclass Example: Dublin Core Resource Record
    7. Data Class as a Code Smell
      1. Data Class as Scaffolding
      2. Data Class as Intermediate Representation
    8. Pattern Matching Class Instances
      1. Simple Class Patterns
      2. Keyword Class Patterns
      3. Positional Class Patterns
    9. Chapter Summary
    10. Further Reading
  8. 6. Object References, Mutability, and Recycling
    1. What’s New in This Chapter
    2. Variables Are Not Boxes
    3. Identity, Equality, and Aliases
      1. Choosing Between == and is
      2. The Relative Immutability of Tuples
    4. Copies Are Shallow by Default
      1. Deep and Shallow Copies of Arbitrary Objects
    5. Function Parameters as References
      1. Mutable Types as Parameter Defaults: Bad Idea
      2. Defensive Programming with Mutable Parameters
    6. del and Garbage Collection
    7. Tricks Python Plays with Immutables
    8. Chapter Summary
    9. Further Reading
  9. II. Functions as Objects
  10. 7. Functions as First-Class Objects
    1. What’s New in This Chapter
    2. Treating a Function Like an Object
    3. Higher-Order Functions
      1. Modern Replacements for map, filter, and reduce
    4. Anonymous Functions
    5. The Nine Flavors of Callable Objects
    6. User-Defined Callable Types
    7. From Positional to Keyword-Only Parameters
      1. Positional-Only Parameters
    8. Packages for Functional Programming
      1. The operator Module
      2. Freezing Arguments with functools.partial
    9. Chapter Summary
    10. Further Reading
  11. 8. Type Hints in Functions
    1. What’s New in This Chapter
    2. About Gradual Typing
    3. Gradual Typing in Practice
      1. Starting with Mypy
      2. Making Mypy More Strict
      3. A Default Parameter Value
      4. Using None as a Default
    4. Types Are Defined by Supported Operations
    5. Types Usable in Annotations
      1. The Any Type
      2. Simple Types and Classes
      3. Optional and Union Types
      4. Generic Collections
      5. Tuple Types
      6. Generic Mappings
      7. Abstract Base Classes
      8. Iterable
      9. Parameterized Generics and TypeVar
      10. Static Protocols
      11. Callable
      12. NoReturn
    6. Annotating Positional Only and Variadic Parameters
    7. Imperfect Typing and Strong Testing
    8. Chapter Summary
    9. Further Reading
  12. 9. Decorators and Closures
    1. What’s New in This Chapter
    2. Decorators 101
    3. When Python Executes Decorators
    4. Registration Decorators
    5. Variable Scope Rules
    6. Closures
    7. The nonlocal Declaration
      1. Variable Lookup Logic
    8. Implementing a Simple Decorator
      1. How It Works
    9. Decorators in the Standard Library
      1. Memoization with functools.cache
      2. Using lru_cache
      3. Single Dispatch Generic Functions
    10. Parameterized Decorators
      1. A Parameterized Registration Decorator
      2. The Parameterized Clock Decorator
      3. A Class-Based Clock Decorator
    11. Chapter Summary
    12. Further Reading
  13. 10. Design Patterns with First-Class Functions
    1. What’s New in This Chapter
    2. Case Study: Refactoring Strategy
      1. Classic Strategy
      2. Function-Oriented Strategy
      3. Choosing the Best Strategy: Simple Approach
      4. Finding Strategies in a Module
    3. Decorator-Enhanced Strategy Pattern
    4. The Command Pattern
    5. Chapter Summary
    6. Further Reading
  14. III. Classes and Protocols
  15. 11. A Pythonic Object
    1. What’s New in This Chapter
    2. Object Representations
    3. Vector Class Redux
    4. An Alternative Constructor
    5. classmethod Versus staticmethod
    6. Formatted Displays
    7. A Hashable Vector2d
    8. Supporting Positional Pattern Matching
    9. Complete Listing of Vector2d, Version 3
    10. Private and “Protected” Attributes in Python
    11. Saving Memory with __slots__
      1. Simple Measure of __slot__ Savings
      2. Summarizing the Issues with __slots__
    12. Overriding Class Attributes
    13. Chapter Summary
    14. Further Reading
  16. 12. Special Methods for Sequences
    1. What’s New in This Chapter
    2. Vector: A User-Defined Sequence Type
    3. Vector Take #1: Vector2d Compatible
    4. Protocols and Duck Typing
    5. Vector Take #2: A Sliceable Sequence
      1. How Slicing Works
      2. A Slice-Aware __getitem__
    6. Vector Take #3: Dynamic Attribute Access
    7. Vector Take #4: Hashing and a Faster ==
    8. Vector Take #5: Formatting
    9. Chapter Summary
    10. Further Reading
  17. 13. Interfaces, Protocols, and ABCs
    1. The Typing Map
    2. What’s New in This Chapter
    3. Two Kinds of Protocols
    4. Programming Ducks
      1. Python Digs Sequences
      2. Monkey Patching: Implementing a Protocol at Runtime
      3. Defensive Programming and “Fail Fast”
    5. Goose Typing
      1. Subclassing an ABC
      2. ABCs in the Standard Library
      3. Defining and Using an ABC
      4. ABC Syntax Details
      5. Subclassing an ABC
      6. A Virtual Subclass of an ABC
      7. Usage of register in Practice
      8. Structural Typing with ABCs
    6. Static Protocols
      1. The Typed double Function
      2. Runtime Checkable Static Protocols
      3. Limitations of Runtime Protocol Checks
      4. Supporting a Static Protocol
      5. Designing a Static Protocol
      6. Best Practices for Protocol Design
      7. Extending a Protocol
      8. The numbers ABCs and Numeric Protocols
    7. Chapter Summary
    8. Further Reading
  18. 14. Inheritance: For Better or for Worse
    1. What’s New in This Chapter
    2. The super() Function
    3. Subclassing Built-In Types Is Tricky
    4. Multiple Inheritance and Method Resolution Order
    5. Mixin Classes
      1. Case-Insensitive Mappings
    6. Multiple Inheritance in the Real World
      1. ABCs Are Mixins Too
      2. ThreadingMixIn and ForkingMixIn
      3. Django Generic Views Mixins
      4. Multiple Inheritance in Tkinter
    7. Coping with Inheritance
      1. Favor Object Composition over Class Inheritance
      2. Understand Why Inheritance Is Used in Each Case
      3. Make Interfaces Explicit with ABCs
      4. Use Explicit Mixins for Code Reuse
      5. Provide Aggregate Classes to Users
      6. Subclass Only Classes Designed for Subclassing
      7. Avoid Subclassing from Concrete Classes
      8. Tkinter: The Good, the Bad, and the Ugly
    8. Chapter Summary
    9. Further Reading
  19. 15. More About Type Hints
    1. What’s New in This Chapter
    2. Overloaded Signatures
      1. Max Overload
      2. Takeaways from Overloading max
    3. TypedDict
    4. Type Casting
    5. Reading Type Hints at Runtime
      1. Problems with Annotations at Runtime
      2. Dealing with the Problem
    6. Implementing a Generic Class
      1. Basic Jargon for Generic Types
    7. Variance
      1. An Invariant Dispenser
      2. A Covariant Dispenser
      3. A Contravariant Trash Can
      4. Variance Review
    8. Implementing a Generic Static Protocol
    9. Chapter Summary
    10. Further Reading
  20. 16. Operator Overloading
    1. What’s New in This Chapter
    2. Operator Overloading 101
    3. Unary Operators
    4. Overloading + for Vector Addition
    5. Overloading * for Scalar Multiplication
    6. Using @ as an Infix Operator
    7. Wrapping-Up Arithmetic Operators
    8. Rich Comparison Operators
    9. Augmented Assignment Operators
    10. Chapter Summary
    11. Further Reading
  21. IV. Control Flow
  22. 17. Iterators, Generators, and Classic Coroutines
    1. What’s New in This Chapter
    2. A Sequence of Words
    3. Why Sequences Are Iterable: The iter Function
      1. Using iter with a Callable
    4. Iterables Versus Iterators
    5. Sentence Classes with __iter__
      1. Sentence Take #2: A Classic Iterator
      2. Don’t Make the Iterable an Iterator for Itself
      3. Sentence Take #3: A Generator Function
      4. How a Generator Works
    6. Lazy Sentences
      1. Sentence Take #4: Lazy Generator
      2. Sentence Take #5: Lazy Generator Expression
    7. When to Use Generator Expressions
    8. An Arithmetic Progression Generator
      1. Arithmetic Progression with itertools
    9. Generator Functions in the Standard Library
    10. Iterable Reducing Functions
    11. Subgenerators with yield from
      1. Reinventing chain
      2. Traversing a Tree
    12. Generic Iterable Types
    13. Classic Coroutines
      1. Example: Coroutine to Compute a Running Average
      2. Returning a Value from a Coroutine
      3. Generic Type Hints for Classic Coroutines
    14. Chapter Summary
    15. Further Reading
  23. 18. with, match, and else Blocks
    1. What’s New in This Chapter
    2. Context Managers and with Blocks
      1. The contextlib Utilities
      2. Using @contextmanager
    3. Pattern Matching in lis.py: A Case Study
      1. Scheme Syntax
      2. Imports and Types
      3. The Parser
      4. The Environment
      5. The REPL
      6. The Evaluator
      7. Procedure: A Class Implementing a Closure
      8. Using OR-patterns
    4. Do This, Then That: else Blocks Beyond if
    5. Chapter Summary
    6. Further Reading
  24. 19. Concurrency Models in Python
    1. What’s New in This Chapter
    2. The Big Picture
    3. A Bit of Jargon
      1. Processes, Threads, and Python’s Infamous GIL
    4. A Concurrent Hello World
      1. Spinner with Threads
      2. Spinner with Processes
      3. Spinner with Coroutines
      4. Supervisors Side-by-Side
    5. The Real Impact of the GIL
      1. Quick Quiz
    6. A Homegrown Process Pool
      1. Process-Based Solution
      2. Understanding the Elapsed Times
      3. Code for the Multicore Prime Checker
      4. Experimenting with More or Fewer Processes
      5. Thread-Based Nonsolution
    7. Python in the Multicore World
      1. System Administration
      2. Data Science
      3. Server-Side Web/Mobile Development
      4. WSGI Application Servers
      5. Distributed Task Queues
    8. Chapter Summary
    9. Further Reading
      1. Concurrency with Threads and Processes
      2. The GIL
      3. Concurrency Beyond the Standard Library
      4. Concurrency and Scalability Beyond Python
  25. 20. Concurrent Executors
    1. What’s New in This Chapter
    2. Concurrent Web Downloads
      1. A Sequential Download Script
      2. Downloading with concurrent.futures
      3. Where Are the Futures?
    3. Launching Processes with concurrent.futures
      1. Multicore Prime Checker Redux
    4. Experimenting with Executor.map
    5. Downloads with Progress Display and Error Handling
      1. Error Handling in the flags2 Examples
      2. Using futures.as_completed
    6. Chapter Summary
    7. Further Reading
  26. 21. Asynchronous Programming
    1. What’s New in This Chapter
    2. A Few Definitions
    3. An asyncio Example: Probing Domains
      1. Guido’s Trick to Read Asynchronous Code
    4. New Concept: Awaitable
    5. Downloading with asyncio and HTTPX
      1. The Secret of Native Coroutines: Humble Generators
      2. The All-or-Nothing Problem
    6. Asynchronous Context Managers
    7. Enhancing the asyncio Downloader
      1. Using asyncio.as_completed and a Thread
      2. Throttling Requests with a Semaphore
      3. Making Multiple Requests for Each Download
    8. Delegating Tasks to Executors
    9. Writing asyncio Servers
      1. A FastAPI Web Service
      2. An asyncio TCP Server
    10. Asynchronous Iteration and Asynchronous Iterables
      1. Asynchronous Generator Functions
      2. Async Comprehensions and Async Generator Expressions
    11. async Beyond asyncio: Curio
    12. Type Hinting Asynchronous Objects
    13. How Async Works and How It Doesn’t
      1. Running Circles Around Blocking Calls
      2. The Myth of I/O-Bound Systems
      3. Avoiding CPU-Bound Traps
    14. Chapter Summary
    15. Further Reading
  27. V. Metaprogramming
  28. 22. Dynamic Attributes and Properties
    1. What’s New in This Chapter
    2. Data Wrangling with Dynamic Attributes
      1. Exploring JSON-Like Data with Dynamic Attributes
      2. The Invalid Attribute Name Problem
      3. Flexible Object Creation with __new__
    3. Computed Properties
      1. Step 1: Data-Driven Attribute Creation
      2. Step 2: Property to Retrieve a Linked Record
      3. Step 3: Property Overriding an Existing Attribute
      4. Step 4: Bespoke Property Cache
      5. Step 5: Caching Properties with functools
    4. Using a Property for Attribute Validation
      1. LineItem Take #1: Class for an Item in an Order
      2. LineItem Take #2: A Validating Property
    5. A Proper Look at Properties
      1. Properties Override Instance Attributes
      2. Property Documentation
    6. Coding a Property Factory
    7. Handling Attribute Deletion
    8. Essential Attributes and Functions for Attribute Handling
      1. Special Attributes that Affect Attribute Handling
      2. Built-In Functions for Attribute Handling
      3. Special Methods for Attribute Handling
    9. Chapter Summary
    10. Further Reading
  29. 23. Attribute Descriptors
    1. What’s New in This Chapter
    2. Descriptor Example: Attribute Validation
      1. LineItem Take #3: A Simple Descriptor
      2. LineItem Take #4: Automatic Naming of Storage Attributes
      3. LineItem Take #5: A New Descriptor Type
    3. Overriding Versus Nonoverriding Descriptors
      1. Overriding Descriptors
      2. Overriding Descriptor Without __get__
      3. Nonoverriding Descriptor
      4. Overwriting a Descriptor in the Class
    4. Methods Are Descriptors
    5. Descriptor Usage Tips
    6. Descriptor Docstring and Overriding Deletion
    7. Chapter Summary
    8. Further Reading
  30. 24. Class Metaprogramming
    1. What’s New in This Chapter
    2. Classes as Objects
    3. type: The Built-In Class Factory
    4. A Class Factory Function
    5. Introducing __init_subclass__
      1. Why __init_subclass__ Cannot Configure __slots__
    6. Enhancing Classes with a Class Decorator
    7. What Happens When: Import Time Versus Runtime
      1. Evaluation Time Experiments
    8. Metaclasses 101
      1. How a Metaclass Customizes a Class
      2. A Nice Metaclass Example
      3. Metaclass Evaluation Time Experiment
    9. A Metaclass Solution for Checked
    10. Metaclasses in the Real World
      1. Modern Features Simplify or Replace Metaclasses
      2. Metaclasses Are Stable Language Features
      3. A Class Can Only Have One Metaclass
      4. Metaclasses Should Be Implementation Details
    11. A Metaclass Hack with __prepare__
    12. Wrapping Up
    13. Chapter Summary
    14. Further Reading
  31. Afterword
    1. Further Reading
  32. Index
  33. About the Author

Product information

  • Title: Fluent Python, 2nd Edition
  • Author(s): Luciano Ramalho
  • Release date: April 2022
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9781492056355