Publishing Python Packages

Book description

Create masterful, maintainable Python packages! This book includes pro tips for design, automation, testing, deployment, and even release as an open source project!

In Publishing Python Packages you will learn how to:

  • Build extensions and console script commands
  • Use tox to automate packaging, installing, and testing
  • Build a continuous integration pipeline using GitHub Actions
  • Improve code quality and reduce manual review using black, mypy, and flake8
  • Create published documentation for your packages
  • Keep packages up to date with pyupgrade and Dependabot
  • Foster an open source community using GitHub features

Publishing Python Packages teaches you how to easily share your Python code with your team and the outside world. Learn a repeatable and highly automated process for package maintenance that’s based on the best practices, tools, and standards of Python packaging. This book walks you through creating a complete package, including a C extension, and guides you all the way to publishing on the Python Package Index. Whether you’re entirely new to Python packaging or looking for optimal ways to maintain and scale your packages, this fast-paced and engaging guide is for you.

About the Technology
Successful Python packages install easily, run flawlessly, and stay reliably up to date. Publishing perfect Python packages requires a rigorous process that supports systematic testing and review, along with excellent documentation. Fortunately, the Python ecosystem includes tools and techniques to automate package creation and publishing.

About the Book
Publishing Python Packages presents a practical process for sharing Python code in an automated and scalable way. Get hands-on experience with the latest packaging tools, and learn the ins and outs of package testing and continuous integration. You’ll even get pro tips for setting up a maintainable open source project, including licensing, documentation, and nurturing a community of contributors.

What's Inside
  • Build extensions and console script commands
  • Improve code quality with automated review and testing
  • Create excellent documentation
  • Keep packages up to date with pyupgrade and Dependabot


About the Reader
For intermediate Python programmers.

About the Author
Dane Hillard has spent the majority of his development career using Python to build web applications.

Quotes
...practical...I learned a lot from this book. A wholly modern approach to current packaging tooling and the problems faced by package developers…I think you’ll find it to be a useful source and an inspiration.
- From the Foreword by David Beazley, author of Python Distilled

A comprehensive guide to the Python packaging landscape. Dane takes a complicated topic and makes you think, ‘Yeah, I got this.’ It’s impressively done. Recommended.
- Carlton Gibson, Django Project

A comprehensive guide for Python programmers ready to share their code with the world.
- Joshua A. McAdams, Senior Software Engineer, Google

Table of contents

  1. inside front cover
  2. Publishing Python Packages
  3. Copyright
  4. brief contents
  5. contents
  6. front matter
    1. foreword
    2. preface
    3. acknowledgments
    4. about this book
    5. Who should read this book?
    6. How this book is organized: a roadmap
    7. About the code
    8. liveBook discussion forum
    9. about the author
    10. about the cover illustration
  7. Part 1. Foundations
  8. 1 The what and why of Python packages
    1. 1.1 What is a package, anyway?
      1. 1.1.1 Standardizing packaging for automation
      2. 1.1.2 The contents of a distribution package
      3. 1.1.3 The challenges of sharing software
    2. 1.2 How packaging helps you
      1. 1.2.1 Enforcing cohesion and encapsulation through packaging
      2. 1.2.2 Promoting clear ownership of code
      3. 1.2.3 Decoupling implementation from usage
      4. 1.2.4 Filling roles by composing small packages
    3. Summary
  9. 2 Preparing for package development
    1. 2.1 Managing Python virtual environments
      1. 2.1.1 Creating virtual environments with venv
    2. Summary
  10. 3 The anatomy of a minimal Python package
    1. 3.1 The Python build workflow
      1. 3.1.1 Parts of the Python build system
    2. 3.2 Authoring package metadata
      1. 3.2.1 Required core metadata
      2. 3.2.2 Optional core metadata
      3. 3.2.3 Specifying a license
    3. 3.3 Controlling source code and file discovery
    4. 3.4 Including non-Python files in a package
    5. Summary
  11. Part 2. Creating a viable package
  12. 4 Handling package dependencies, entry points, and extensions
    1. 4.1 A package for calculating vehicle drift
    2. 4.2 Creating a C extension for Python
      1. 4.2.1 Creating C extension source code
      2. 4.2.2 Integrating Cython into a Python package build
      3. 4.2.3 Installing and profiling your C extension
      4. 4.2.4 Build targets for binary wheel distributions
      5. 4.2.5 Specifying required Python versions
    3. 4.3 Offering command-line tools from a Python package
      1. 4.3.1 Creating commands with Setuptools entry points
    4. 4.4 Specifying dependencies for Python packages
    5. Answer to exercises
    6. Summary
  13. 5 Building and maintaining a test suite
    1. 5.1 Integrating a testing setup
      1. 5.1.1 The pytest testing framework
      2. 5.1.2 Adding test coverage measurement
      3. 5.1.3 Increasing test coverage
    2. 5.2 Addressing testing tedium
      1. 5.2.1 Addressing repetitive, data-driven tests
      2. 5.2.2 Addressing frequent package installation
      3. 5.2.3 Configuring test environments
      4. 5.2.4 Tips for quicker and safer testing
    3. Answers to exercises
    4. Summary
  14. 6 Automating code quality tooling
    1. 6.1 The true power of tox environments
      1. 6.1.1 Creating nondefault tox environments
      2. 6.1.2 Managing dependencies across tox environments
    2. 6.2 Analyzing type safety
      1. 6.2.1 Creating a tox environment for type checking
      2. 6.2.2 Configuring mypy
    3. 6.3 Creating a tox environment for code formatting
      1. 6.3.1 Configuring black
    4. 6.4 Creating a tox environment for linting
      1. 6.4.1 Configuring flake8
    5. Answers to exercises
    6. Summary
  15. Part 3. Going public
  16. 7 Automating work through continuous integration
    1. 7.1 The continuous integration workflow
    2. 7.2 Continuous integration with GitHub Actions
      1. 7.2.1 A high-level GitHub Actions workflow
      2. 7.2.2 Understanding GitHub Actions terminology
      3. 7.2.3 Starting a GitHub Actions workflow configuration
    3. 7.3 Converting manual tasks to GitHub Actions
      1. 7.3.1 Running a job multiple times with a build matrix
      2. 7.3.2 Building Python package distributions for a variety of platforms
    4. 7.4 Publishing a package
    5. Summary
  17. 8 Authoring and maintaining documentation
    1. 8.1 Some quick philosophy on documentation
    2. 8.2 Starting your documentation with Sphinx
      1. 8.2.1 Automating documentation refresh during development
      2. 8.2.2 Automating extraction of code documentation
    3. 8.3 Publishing documentation to Read the Docs
      1. 8.3.1 Configuring Read the Docs
    4. 8.4 Documentation best practices
      1. 8.4.1 What to document
      2. 8.4.2 Prefer linking over repetition
      3. 8.4.3 Use consistent, empathetic language
      4. 8.4.4 Avoid assumptions and create context
      5. 8.4.5 Create visual interest and coherent structure
      6. 8.4.6 Powering up your documentation
    5. Summary
  18. 9 Making a package evergreen
    1. 9.1 Choosing a package-versioning strategy
    2. 9.1.1 Direct and indirect dependencies
    3. 9.1.2 Python dependency specifiers and dependency hell
    4. 9.1.3 Semantic versioning and calendar versioning
    5. 9.2 Getting the most out of GitHub
    6. 9.2.1 The GitHub dependency graph
    7. 9.2.2 Mitigating security vulnerabilities with Dependabot
    8. 9.3 Thresholding test coverage
    9. 9.4 Updating Python syntax using pyupgrade
    10. 9.5 Reducing rework using pre-commit hooks
    11. Answers to exercises
    12. Summary
  19. Part 4. The long haul
  20. 10 Scaling and solidifying your practices
    1. 10.1 Creating a project template for future packages
      1. 10.1.1 Creating a cookiecutter configuration
      2. 10.1.2 Extracting a cookiecutter template from an existing project
    2. 10.2 Using namespace packages
      1. 10.2.1 Converting an existing package to a namespace package
    3. 10.3 Scaling packaging in your organization
      1. 10.3.1 Private package repository servers
    4. Summary
  21. 11 Building a community
    1. 11.1 Your README needs to make a value proposition
    2. 11.2 Provide supporting documentation for different user types
    3. 11.3 Establish, provide, and enforce a code of conduct
    4. 11.4 Conveying the project’s road map, status, and changes
    5. 11.4.1 Using GitHub projects for kanban management
    6. 11.4.2 Use GitHub labels to track status for individual tasks
    7. 11.4.3 Track high-level changes in a log
    8. 11.5 Gather consistent information with issue templates
    9. 11.6 Go forth
    10. Summary
  22. Appendix A. Installing asdf and python-launcher
    1. A.1 Installing asdf
    2. A.2 Installing python-launcher
    3. Answer to exercise A.1
  23. Appendix B. Installing pipx, build, tox, pre-commit, and cookiecutter
    1. B.1 Installing pipx
    2. B.2 Installing build
    3. B.3 Installing tox
    4. B.4 Installing pre-commit
    5. B.5 Installing cookiecutter
  24. index
  25. inside back cover

Product information

  • Title: Publishing Python Packages
  • Author(s): Dane Hillard
  • Release date: February 2023
  • Publisher(s): Manning Publications
  • ISBN: 9781617299919