O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

High Performance Python, 2nd Edition

Book Description

Your Python code may run correctly, but you need it to run faster. By exploring the fundamental theory behind design choices, the updated edition of this practical guide, expanded and enhanced for Python 3, helps you gain a deeper understanding of Python’s implementation. You’ll learn how to locate performance bottlenecks and significantly speed up your code in high-data-volume programs.

How can you take advantage of multicore architectures or clusters? Or build a system that can scale up and down without losing reliability? Experienced Python programmers and students alike will learn concrete solutions to these and other issues, along with war stories from companies that use high-performance Python for social media analytics, productionized machine learning, and other situations.

  • Get a better grasp of NumPy, Cython, and profilers
  • Learn how Python abstracts the underlying computer architecture
  • Understand performant pandas
  • Use profiling to find bottlenecks in CPU time and memory usage
  • Write efficient programs by choosing appropriate data structures
  • Speed up matrix, vector, and even tensor computations
  • Use tools to compile Python down to machine code, on CPUs and GPUs
  • Manage multiple I/O and computational operations concurrently
  • Convert multiprocessing code to run on a local or remote cluster
  • Deploy code faster using tools like Docker
  • Solve large problems while using less RAM
  • Get real-life stories and lessons from Python programmers

Table of Contents

  1. Preface
    1. Who This Book Is For
    2. Who This Book Is Not For
    3. What You’ll Learn
    4. Python 3
    5. Changes from Python 2.7
    6. License
    7. How to Make an Attribution
    8. Errata and Feedback
    9. Conventions Used in This Book
    10. Using Code Examples
    11. Safari® Books Online
    12. How to Contact Us
    13. Acknowledgments
  2. 1. Understanding Performant Python
    1. The Fundamental Computer System
      1. Computing Units
      2. Memory Units
      3. Communications Layers
    2. Putting the Fundamental Elements Together
      1. Idealized Computing Versus the Python Virtual Machine
    3. So Why Use Python?
    4. How to be a Highly Performant Programmer
      1. Good working practices
      2. Some Thoughts on Good Notebook Practice
      3. Getting the joy back into your work
  3. 2. Profiling to Find Bottlenecks
    1. Profiling Efficiently
    2. Introducing the Julia Set
    3. Calculating the Full Julia Set
    4. Simple Approaches to Timing—print and a Decorator
    5. Simple Timing Using the Unix time Command
    6. Using the cProfile Module
    7. Visualizing cProfile Output with SnakeViz
    8. Using line_profiler for Line-by-Line Measurements
    9. Using memory_profiler to Diagnose Memory Usage
    10. Introspecting an Existing Process with PySpy
    11. Using the dis Module to Examine CPython Bytecode
      1. Different Approaches, Different Complexity
    12. Unit Testing During Optimization to Maintain Correctness
      1. No-op @profile Decorator
    13. Strategies to Profile Your Code Successfully
    14. Wrap-Up
  4. 3. Lists and Tuples
    1. A More Efficient Search
    2. Lists Versus Tuples
      1. Lists as Dynamic Arrays
      2. Tuples As Static Arrays
    3. Wrap-Up
  5. 4. Dictionaries and Sets
    1. How Do Dictionaries and Sets Work?
      1. Inserting and Retrieving
      2. Deletion
      3. Resizing
      4. Hash Functions and Entropy
    2. Dictionaries and Namespaces
    3. Wrap-Up