Expert Python Programming - Second Edition

Expert Python Programming - Second Edition

by Michał Jaworski, Tarek Ziadé
Released May 2016
Publisher(s): Packt Publishing
ISBN: 9781785886850

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Become an ace Python programmer by learning best coding practices and advance-level concepts with Python 3.5

About This Book

  • Based on the latest stable version of Python (version 3.5)
  • Creating well manageable code that will run in various environments with different sets of dependencies
  • Packed with advanced concepts and best practices to write efficient Python code

Who This Book Is For

The book would appeal to web developers and Python programmers who want to start using version 3.5 and write code efficiently. Basic knowledge of Python programming is expected.

What You Will Learn

  • Conventions and best practices that are widely adopted in the python community
  • Package python code effectively for community and production use
  • Easy and lightweight ways to automate code deployment on remote systems
  • Improve your code's quality, reliability, and performance
  • Write concurrent code in python
  • Extend python with code written in different languages

In Detail

Python is a dynamic programming language, used in a wide range of domains by programmers who find it simple, yet powerful. Even if you find writing Python code easy, writing code that is efficient and easy to maintain and reuse is a challenge.

The focus of the book is to familiarize you with common conventions, best practices, useful tools and standards used by python professionals on a daily basis when working with code.

You will begin with knowing new features in Python 3.5 and quick tricks for improving productivity. Next, you will learn advanced and useful python syntax elements brought to this new version. Using advanced object-oriented concepts and mechanisms available in python, you will learn different approaches to implement metaprogramming. You will learn to choose good names, write packages, and create standalone executables easily.

You will also be using some powerful tools such as buildout and vitualenv to release and deploy the code on remote servers for production use. Moving on, you will learn to effectively create Python extensions with C, C++, cython, and pyrex. The important factors while writing code such as code management tools, writing clear documentation, and test-driven development are also covered.

You will now dive deeper to make your code efficient with general rules of optimization, strategies for finding bottlenecks, and selected tools for application optimization.

By the end of the book, you will be an expert in writing efficient and maintainable code.

Style and approach

An easy-to-follow guide that covers industry followed best practices in Python programming

Publisher resources

Download Example Code

Table of contents

  1. Expert Python Programming Second Edition
    1. Table of Contents
    2. Expert Python Programming Second Edition
    3. Credits
    4. About the Authors
    5. About the Reviewer
    6. www.PacktPub.com
      1. eBooks, discount offers, and more
        1. Why subscribe?
    7. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Errata
        3. Piracy
        4. Questions
    8. 1. Current Status of Python
      1. Where are we now and where we are going?
      2. Why and how does Python change?
      3. Getting up to date with changes – PEP documents
      4. Python 3 adoption at the time of writing this book
      5. The main differences between Python 3 and Python 2
        1. Why should I care?
        2. The main syntax differences and common pitfalls
          1. Syntax changes
          2. Changes in the standard library
          3. Changes in datatypes and collections
        3. The popular tools and techniques used for maintaining cross-version compatibility
      6. Not only CPython
        1. Why should I care?
        2. Stackless Python
        3. Jython
        4. IronPython
        5. PyPy
      7. Modern approaches to Python development
      8. Application-level isolation of Python environments
        1. Why isolation?
        2. Popular solutions
          1. virtualenv
          2. venv
          3. buildout
        3. Which one to choose?
      9. System-level environment isolation
        1. Virtual development environments using Vagrant
        2. Containerization versus virtualization
      10. Popular productivity tools
        1. Custom Python shells – IPython, bpython, ptpython, and so on
          1. Setting up the PYTHONSTARTUP environment variable
          2. IPython
          3. bpython
          4. ptpython
        2. Interactive debuggers
      11. Useful resources
      12. Summary
    9. 2. Syntax Best Practices – below the Class Level
      1. Python's built-in types
        1. Strings and bytes
          1. Implementation details
          2. String concatenation
        2. Collections
          1. Lists and tuples
            1. Implementation details
            2. List comprehensions
            3. Other idioms
          2. Dictionaries
            1. Implementation details
            2. Weaknesses and alternatives
          3. Sets
            1. Implementation details
          4. Beyond basic collections – the collections module
      2. Advanced syntax
        1. Iterators
        2. The yield statement
        3. Decorators
          1. General syntax and possible implementations
            1. As a function
            2. As a class
            3. Parametrizing decorators
            4. Introspection preserving decorators
          2. Usage and useful examples
            1. Argument checking
            2. Caching
            3. Proxy
            4. Context provider
        4. Context managers – the with statement
          1. General syntax and possible implementations
            1. As a class
            2. As a function – the contextlib module
      3. Other syntax elements you may not know yet
        1. The for … else … statement
        2. Function annotations
          1. The general syntax
          2. The possible uses
      4. Summary
    10. 3. Syntax Best Practices – above the Class Level
      1. Subclassing built-in types
      2. Accessing methods from superclasses
        1. Old-style classes and super in Python 2
        2. Understanding Python's Method Resolution Order
        3. super pitfalls
          1. Mixing super and explicit class calls
          2. Heterogeneous arguments
        4. Best practices
      3. Advanced attribute access patterns
        1. Descriptors
          1. Real-life example – lazily evaluated attributes
        2. Properties
        3. Slots
      4. Metaprogramming
        1. Decorators – a method of metaprogramming
        2. Class decorators
        3. Using the __new__() method to override instance creation process
        4. Metaclasses
          1. The general syntax
          2. New Python 3 syntax for metaclasses
          3. Metaclass usage
          4. Metaclass pitfalls
        5. Some tips on code generation
          1. exec, eval, and compile
          2. Abstract Syntax Tree
            1. Import hooks
          3. Projects using code generation patterns
            1. Falcon's compiled router
            2. Hy
      5. Summary
    11. 4. Choosing Good Names
      1. PEP 8 and naming best practices
        1. Why and when to follow PEP 8?
        2. Beyond PEP 8 – team-specific style guidelines
      2. Naming styles
        1. Variables
          1. Constants
          2. Naming and usage
          3. Public and private variables
          4. Functions and methods
          5. The private controversy
          6. Special methods
          7. Arguments
          8. Properties
          9. Classes
          10. Modules and packages
      3. The naming guide
        1. Using the has or is prefix for Boolean elements
        2. Using plurals for variables that are collections
        3. Using explicit names for dictionaries
        4. Avoiding generic names
        5. Avoiding existing names
      4. Best practices for arguments
        1. Building arguments by iterative design
        2. Trust the arguments and your tests
        3. Using *args and **kwargs magic arguments carefully
      5. Class names
      6. Module and package names
      7. Useful tools
        1. Pylint
        2. pep8 and flake8
      8. Summary
    12. 5. Writing a Package
      1. Creating a package
        1. The confusing state of Python packaging tools
          1. The current landscape of Python packaging thanks to PyPA
          2. Tool recommendations
        2. Project configuration
          1. setup.py
          2. setup.cfg
          3. MANIFEST.in
          4. Most important metadata
          5. Trove classifiers
          6. Common patterns
            1. Automated inclusion of version string from package
            2. README file
            3. Managing dependencies
        3. The custom setup command
        4. Working with packages during development
          1. setup.py install
          2. Uninstalling packages
          3. setup.py develop or pip -e
      2. Namespace packages
        1. Why is it useful?
        2. PEP 420 – implicit namespace packages
        3. Namespace packages in previous Python versions
      3. Uploading a package
        1. PyPI – Python Package Index
          1. Uploading to PyPI – or other package index
          2. .pypirc
        2. Source packages versus built packages
          1. sdist
          2. bdist and wheels
      4. Standalone executables
        1. When are standalone executables useful?
        2. Popular tools
          1. PyInstaller
          2. cx_Freeze
          3. py2exe and py2app
        3. Security of Python code in executable packages
          1. Making decompilation harder
      5. Summary
    13. 6. Deploying Code
      1. The Twelve-Factor App
      2. Deployment automation using Fabric
      3. Your own package index or index mirror
        1. PyPI mirroring
        2. Deployment using a package
      4. Common conventions and practices
        1. The filesystem hierarchy
        2. Isolation
        3. Using process supervision tools
        4. Application code should be run in user space
        5. Using reverse HTTP proxies
        6. Reloading processes gracefully
      5. Code instrumentation and monitoring
        1. Logging errors – sentry/raven
        2. Monitoring system and application metrics
        3. Dealing with application logs
          1. Basic low-level log practices
          2. Tools for log processing
      6. Summary
    14. 7. Python Extensions in Other Languages
      1. Different language means – C or C++
        1. How do extensions in C or C++ work
      2. Why you might want to use extensions
        1. Improving performance in critical code sections
        2. Integrating existing code written in different languages
        3. Integrating third-party dynamic libraries
        4. Creating custom datatypes
      3. Writing extensions
        1. Pure C extensions
          1. A closer look at Python/C API
          2. Calling and binding conventions
          3. Exception handling
          4. Releasing GIL
          5. Reference counting
        2. Cython
          1. Cython as a source to source compiler
          2. Cython as a language
      4. Challenges
        1. Additional complexity
        2. Debugging
      5. Interfacing with dynamic libraries without extensions
        1. ctypes
          1. Loading libraries
          2. Calling C functions using ctypes
          3. Passing Python functions as C callbacks
        2. CFFI
      6. Summary
    15. 8. Managing Code
      1. Version control systems
        1. Centralized systems
        2. Distributed systems
          1. Distributed strategies
        3. Centralized or distributed?
        4. Use Git if you can
        5. Git flow and GitHub flow
      2. Continuous development processes
        1. Continuous integration
          1. Testing every commit
          2. Merge testing through CI
          3. Matrix testing
        2. Continuous delivery
        3. Continuous deployment
        4. Popular tools for continuous integration
          1. Jenkins
          2. Buildbot
          3. Travis CI
          4. GitLab CI
        5. Choosing the right tool and common pitfalls
          1. Problem 1 – too complex build strategies
          2. Problem 2 – too long building time
          3. Problem 3 – external job definitions
          4. Problem 4 – lack of isolation
      3. Summary
    16. 9. Documenting Your Project
      1. The seven rules of technical writing
        1. Write in two steps
        2. Target the readership
        3. Use a simple style
        4. Limit the scope of information
        5. Use realistic code examples
        6. Use a light but sufficient approach
        7. Use templates
      2. A reStructuredText primer
        1. Section structure
        2. Lists
        3. Inline markup
        4. Literal block
        5. Links
      3. Building the documentation
        1. Building the portfolio
          1. Design
          2. Usage
            1. Recipe
            2. Tutorial
            3. Module helper
          3. Operations
      4. Making your own portfolio
        1. Building the landscape
          1. Producer's layout
          2. Consumer's layout
            1. Working on the index pages
            2. Registering module helpers
            3. Adding index markers
            4. Cross-references
        2. Documentation building and continuous integration
      5. Summary
    17. 10. Test-Driven Development
      1. I don't test
        1. Test-driven development principles
          1. Preventing software regression
          2. Improving code quality
          3. Providing the best developer documentation
          4. Producing robust code faster
        2. What kind of tests?
          1. Acceptance tests
          2. Unit tests
          3. Functional tests
          4. Integration tests
          5. Load and performance testing
          6. Code quality testing
        3. Python standard test tools
          1. unittest
          2. doctest
      2. I do test
        1. unittest pitfalls
        2. unittest alternatives
          1. nose
            1. Test runner
            2. Writing tests
            3. Writing test fixtures
            4. Integration with setuptools and a plug-in system
            5. Wrap-up
          2. py.test
            1. Writing test fixtures
            2. Disabling test functions and classes
            3. Automated distributed tests
            4. Wrap-up
        3. Testing coverage
        4. Fakes and mocks
          1. Building a fake
          2. Using mocks
        5. Testing environment and dependency compatibility
          1. Dependency matrix testing
        6. Document-driven development
          1. Writing a story
      3. Summary
    18. 11. Optimization – General Principles and Profiling Techniques
      1. The three rules of optimization
        1. Make it work first
        2. Work from the user's point of view
        3. Keep the code readable and maintainable
      2. Optimization strategy
        1. Find another culprit
        2. Scale the hardware
        3. Writing a speed test
      3. Finding bottlenecks
        1. Profiling CPU usage
          1. Macro-profiling
          2. Micro-profiling
          3. Measuring Pystones
        2. Profiling memory usage
          1. How Python deals with memory
          2. Profiling memory
            1. objgraph
          3. C code memory leaks
        3. Profiling network usage
      4. Summary
    19. 12. Optimization – Some Powerful Techniques
      1. Reducing the complexity
        1. Cyclomatic complexity
        2. The big O notation
      2. Simplifying
        1. Searching in a list
          1. Using a set instead of a list
            1. Cut the external calls, reduce the workload
      3. Using collections
        1. deque
        2. defaultdict
        3. namedtuple
      4. Using architectural trade-offs
        1. Using heuristics and approximation algorithms
        2. Using task queues and delayed processing
        3. Using probabilistic data structures
      5. Caching
        1. Deterministic caching
        2. Nondeterministic caching
        3. Cache services
          1. Memcached
      6. Summary
    20. 13. Concurrency
      1. Why concurrency?
      2. Multithreading
        1. What is multithreading?
        2. How Python deals with threads
        3. When should threading be used?
          1. Building responsive interfaces
          2. Delegating work
          3. Multiuser applications
          4. An example of a threaded application
            1. Using one thread per item
            2. Using a thread pool
            3. Using two-way queues
            4. Dealing with errors and rate limiting
      3. Multiprocessing
        1. The built-in multiprocessing module
          1. Using process pools
          2. Using multiprocessing.dummy as a multithreading interface
      4. Asynchronous programming
        1. Cooperative multitasking and asynchronous I/O
        2. Python async and await keywords
        3. asyncio in older versions of Python
        4. A practical example of asynchronous programming
        5. Integrating nonasynchronous code with async using futures
          1. Executors and futures
          2. Using executors in an event loop
      5. Summary
    21. 14. Useful Design Patterns
      1. Creational patterns
        1. Singleton
      2. Structural patterns
        1. Adapter
          1. Interfaces
            1. Using zope.interface
            2. Using function annotations and abstract base classes
            3. Using collections.abc
        2. Proxy
        3. Facade
      3. Behavioral patterns
        1. Observer
        2. Visitor
        3. Template
      4. Summary
    22. Index

Product information

  • Title: Expert Python Programming - Second Edition
  • Author(s): Michał Jaworski, Tarek Ziadé
  • Release date: May 2016
  • Publisher(s): Packt Publishing
  • ISBN: 9781785886850