Python Architecture Patterns

Table of contents

1. Preface
   1. Who this book is for
   2. What this book covers
   3. To get the most out of this book
   4. Get in touch
   5. Share your thoughts
2. Introduction to Software Architecture
   1. Defining the structure of a system
   2. Division into smaller units
      1. In-process communication
   3. Conway's Law – Effects on software architecture
   4. Application example – Overview
   5. Security aspects of software architecture
   6. Summary
3. Part I
4. Design
5. API Design
   1. Abstractions
      1. Using the right abstractions
      2. Leaking abstractions
      3. Resources and action abstractions
   2. RESTful interfaces
      1. A more practical definition
      2. Headers and statuses
      3. Designing resources
      4. Resources and parameters
      5. Pagination
      6. Designing a RESTful API process
      7. Using the Open API specification
   3. Authentication
      1. Authenticating HTML interfaces
      2. Authenticating RESTful interfaces
         1. Self-encoded tokens
   4. Versioning the API
      1. Why versioning?
      2. Internal versus external versioning
      3. Semantic versioning
      4. Simple versioning
   5. Frontend and backend
      1. Model View Controller structure
   6. HTML interfaces
      1. Traditional HTML interfaces
      2. Dynamic pages
      3. Single-page apps
      4. Hybrid approach
   7. Designing the API for the example
      1. Endpoints
      2. Review of the design and implementation
   8. Summary
6. Data Modeling
   1. Types of databases
      1. Relational databases
      2. Non-relational databases
         1. Key-value stores
         2. Document stores
         3. Wide-column databases
         4. Graph databases
      3. Small databases
   2. Database transactions
   3. Distributed relational databases
      1. Primary/replica
      2. Sharding
         1. Pure sharding
         2. Mixed sharding
         3. Table sharding
      3. Advantages and disadvantages of sharding
   4. Schema design
      1. Schema normalization
      2. Denormalization
   5. Data indexing
      1. Cardinality
   6. Summary
7. The Data Layer
   1. The Model layer
      1. Domain-Driven Design
      2. Using ORM
         1. Independence from the database
         2. Independence from SQL and the Repository pattern
         3. No problems related to composing SQL
      3. The Unit of Work pattern and encapsulating the data
      4. CQRS, using different models for read and write
   2. Database migrations
      1. Backward compatibility
      2. Relational schema changes
         1. Changing the database without interruption
         2. Data migrations
      3. Changes without enforcing a schema
   3. Dealing with legacy databases
      1. Detecting a schema from a database
      2. Syncing the existing schema to the ORM definition
   4. Summary
8. Part II
9. Architectural Patterns
10. The Twelve-Factor App Methodology
    1. Intro to the Twelve-Factor App
    2. Continuous Integration
    3. Scalability
    4. Configuration
    5. The Twelve Factors
       1. Build once, run multiple times
       2. Dependencies and configurations
       3. Scalability
       4. Monitoring and admin
    6. Containerized Twelve-Factor Apps
    7. Summary
11. Web Server Structures
    1. Request-response
    2. Web architecture
    3. Web servers
       1. Serving static content externally
       2. Reverse proxy
       3. Logging
       4. Advanced usages
    4. uWSGI
       1. The WSGI application
       2. Interacting with the web server
       3. Processes
       4. Process lifecycle
    5. Python worker
       1. Django MVT architecture
       2. Routing a request towards a View
       3. The View
          1. HttpRequest
          2. HttpResponse
       4. Middleware
       5. Django REST framework
          1. Models
          2. URL routing
          3. Views
          4. Serializer
    6. External layers
    7. Summary
12. Event-Driven Structures
    1. Sending events
    2. Asynchronous tasks
    3. Subdividing tasks
    4. Scheduled tasks
    5. Queue effects
       1. Single code for all workers
       2. Cloud queues and workers
    6. Celery
       1. Configuring Celery
       2. Celery worker
       3. Triggering tasks
       4. Connecting the dots
       5. Scheduled tasks
       6. Celery Flower
       7. Flower HTTP API
    7. Summary
13. Advanced Event-Driven Structures
    1. Streaming events
    2. Pipelines
       1. Preparation
       2. Base task
       3. Image task
       4. Video task
       5. Connecting the tasks
       6. Running the task
    3. Defining a bus
    4. More complex systems
    5. Testing event-driven systems
    6. Summary
14. Microservices vs Monolith
    1. Monolithic architecture
    2. The microservices architecture
    3. Which architecture to choose
       1. A side note about similar designs
    4. The key factor – team communication
    5. Moving from a monolith to microservices
       1. Challenges for the migration
       2. A move in four acts
          1. 1. Analyze
          2. 2. Design
          3. 3. Plan
          4. 4. Execute
    6. Containerizing services
       1. Building and running an image
       2. Building and running a web service
          1. uWSGI configuration
          2. nginx configuration
          3. Start script
          4. Building and running
          5. Caveats
    7. Orchestration and Kubernetes
    8. Summary
15. Part III
16. Implementation
17. Testing and TDD
    1. Testing the code
    2. Different levels of testing
       1. Unit tests
       2. Integration tests
       3. System tests
    3. Testing philosophy
       1. How to design a great test
       2. Structuring tests
    4. Test-Driven Development
       1. Introducing TDD into new teams
       2. Problems and limitations
       3. Example of the TDD process
    5. Introduction to unit testing in Python
       1. Python unittest
       2. Pytest
    6. Testing external dependencies
       1. Mocking
       2. Dependency injection
       3. Dependency injection in OOP
    7. Advanced pytest
       1. Grouping tests
       2. Using fixtures
    8. Summary
18. Package Management
    1. The creation of a new package
    2. Trivial packaging in Python
    3. The Python packaging ecosystem
       1. PyPI
       2. Virtual environments
       3. Preparing an environment
          1. A note on containers
       4. Python packages
    4. Creating a package
       1. Development mode
       2. Pure Python package
    5. Cython
    6. Python package with binary code
    7. Uploading your package to PyPI
    8. Creating your own private index
    9. Summary
19. Part IV
20. Ongoing operations
21. Logging
    1. Log basics
    2. Producing logs in Python
    3. Detecting problems through logs
       1. Detecting expected errors
       2. Capturing unexpected errors
    4. Log strategies
    5. Adding logs while developing
    6. Log limitations
    7. Summary
22. Metrics
    1. Metrics versus logs
       1. Kinds of metrics
    2. Generating metrics with Prometheus
       1. Preparing the environment
       2. Configuring Django Prometheus
       3. Checking the metrics
       4. Starting a Prometheus server
    3. Querying Prometheus
    4. Proactively working with metrics
    5. Alerting
    6. Summary
23. Profiling
    1. Profiling basics
    2. Types of profilers
    3. Profiling code for time
       1. Using the built-in cProfile module
       2. Line profiler
    4. Partial profiling
       1. Example web server returning prime numbers
       2. Profiling the whole process
       3. Generating a profile file per request
    5. Memory profiling
       1. Using memory_profiler
       2. Memory optimization
    6. Summary
24. Debugging
    1. Detecting and processing defects
    2. Investigation in production
    3. Understanding the problem in production
       1. Logging a request ID
       2. Analyzing data
       3. Increasing logging
    4. Local debugging
    5. Python introspection tools
    6. Debugging with logs
    7. Debugging with breakpoints
    8. Summary
25. Ongoing Architecture
    1. Adjusting the architecture
    2. Scheduled downtime
       1. Maintenance window
    3. Incidents
       1. Postmortem analysis
       2. Premortem analysis
    4. Load testing
    5. Versioning
    6. Backward compatibility
       1. Incremental changes
       2. Deploying without interruption
    7. Feature flags
    8. Teamwork aspects of changes
    9. Summary
26. Other Books You May Enjoy
27. Index