O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Software Architecture with Python

Book Description

Architect and design highly scalable, robust, clean, and highly performant applications in Python

About This Book

  • Identify design issues and make the necessary adjustments to achieve improved performance
  • Understand practical architectural quality attributes from the perspective of a practicing engineer and architect using Python
  • Gain knowledge of architectural principles and how they can be used to provide accountability and rationale for architectural decisions

Who This Book Is For

This book is for experienced Python developers who are aspiring to become the architects of enterprise-grade applications or software architects who would like to leverage Python to create effective blueprints of applications.

What You Will Learn

  • Build programs with the right architectural attributes
  • Use Enterprise Architectural Patterns to solve scalable problems on the Web
  • Understand design patterns from a Python perspective
  • Optimize the performance testing tools in Python
  • Deploy code in remote environments or on the Cloud using Python
  • Secure architecture applications in Python

In Detail

This book starts off by explaining how Python fits into an application architecture. As you move along, you will understand the architecturally significant demands and how to determine them. Later, you’ll get a complete understanding of the different architectural quality requirements that help an architect to build a product that satisfies business needs, such as maintainability/reusability, testability, scalability, performance, usability, and security.

You will use various techniques such as incorporating DevOps, Continuous Integration, and more to make your application robust. You will understand when and when not to use object orientation in your applications. You will be able to think of the future and design applications that can scale proportionally to the growing business.

The focus is on building the business logic based on the business process documentation and which frameworks are to be used when. We also cover some important patterns that are to be taken into account while solving design problems as well as those in relatively new domains such as the Cloud.

This book will help you understand the ins and outs of Python so that you can make those critical design decisions that not just live up to but also surpass the expectations of your clients.

Style and approach

Filled with examples and use cases, this guide takes a no-nonsense approach to help you with everything it takes to become a successful software architect.

Downloading the example code for this book. You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the code file.

Table of Contents

  1. Software Architecture with Python
    1. Table of Contents
    2. Software Architecture with Python
    3. Credits
    4. About the Author
    5. About the Reviewer
    6. www.PacktPub.com
    7. Customer Feedback
    8. Preface
      1. What this book covers
      2. What you need for this book
      3. Who this book is for
      4. Conventions
      5. Reader feedback
      6. Customer support
        1. Downloading the example code
        2. Downloading the color images of this book
        3. Errata
        4. Piracy
        5. Questions
    9. 1. Principles of Software Architecture
      1. Defining Software Architecture
        1. Software Architecture versus design
        2. Aspects of Software Architecture
      2. Characteristics of Software Architecture
        1. An architecture defines a structure
        2. An architecture picks a core set of elements
        3. An architecture captures early design decisions
        4. An architecture manages stakeholder requirements
        5. An architecture influences the organizational structure
        6. An architecture is influenced by its environment
        7. An architecture documents the system
        8. An architecture often conforms to a pattern
      3. Importance of Software Architecture
      4. System versus enterprise architecture
      5. Architectural quality attributes
        1. Modifiability
        2. Testability
        3. Scalability
        4. Performance
        5. Availability
        6. Security
        7. Deployability
      6. Summary
    10. 2. Writing Modifiable and Readable Code
      1. What is modifiability?
      2. Aspects related to Modifiability
      3. Understanding readability
        1. Python and readability
        2. Readability - antipatterns
        3. Techniques for readability
        4. Document your code
        5. Follow coding and style guidelines
        6. Review and refactor code
        7. Commenting the code
      4. Fundamentals of Modifiability – Cohesion & Coupling
        1. Measuring cohesion and coupling
        2. Measuring cohesion and coupling – string and text processing
      5. Exploring strategies for modifiability
        1. Providing explicit interfaces
        2. Reducing two-way dependencies
        3. Abstract common services
        4. Using inheritance techniques
        5. Using late binding techniques
      6. Metrics – tools for static analysis
        1. What are code smells?
        2. Cyclomatic complexity – the McCabe metric
          1. Testing for metrics
        3. Running Static Checkers
      7. Refactoring Code
        1. Refactoring code – fixing complexity
        2. Refactoring code - fixing code smells
        3. Refactoring code - fixing styling and coding issues
      8. Summary
    11. 3. Testability – Writing Testable Code
      1. Understanding testability
        1. Software testability and related attributes
        2. Testability – architectural aspects
        3. Testability – strategies
          1. Reduce system complexity
          2. Improving predictability
          3. Control and isolate external dependencies
      2. White-box testing principles
        1. Unit testing
        2. Unit testing in action
          1. Extending our unit test case
        3. Nosing around with nose2
        4. Testing with py.test
        5. Code coverage
          1. Measuring coverage using coverage.py
          2. Measuring coverage using nose2
          3. Measuring coverage using py.test
        6. Mocking things up
        7. Tests inline in documentation – doctests
        8. Integration tests
        9. Test automation
          1. Test automation using Selenium Web Driver
        10. Test-Driven Development
        11. TDD with palindromes
        12. Summary
    12. 4. Good Performance is Rewarding!
      1. What is performance?
      2. Software performance engineering
      3. Performance testing and measurement tools
      4. Performance complexity
      5. Measuring performance
        1. Measuring time using a context manager
        2. Timing code using the timeit module
          1. Measuring the performance of our code using timeit
        3. Finding out time complexity – graphs
        4. Measuring CPU time with timeit
      6. Profiling
        1. Deterministic profiling
        2. Profiling with cProfile and profile
          1. Prime number iterator class – performance tweaks
        3. Profiling – collecting and reporting statistics
        4. Third-party profilers
          1. Line profiler
          2. Memory profiler
          3. Substring (subsequence) problem
      7. Other tools
        1. Objgraph
        2. Pympler
      8. Programming for performance – data structures
        1. Mutable containers – lists, dictionaries, and sets
          1. Lists
          2. Dictionaries
          3. Sets
        2. Immutable containers – tuples
        3. High performance containers – the collections module
          1. deque
          2. defaultdict
          3. OrderedDict
            1. Dropping duplicates from a container without losing the order
            2. Implementing a Least Recently Used (LRU) cache dictionary
          4. Counter
          5. ChainMap
          6. namedtuple
        4. Probabilistic data structures – bloom filters
      9. Summary
    13. 5. Writing Applications That Scale
      1. Scalability and performance
      2. Concurrency
        1. Concurrency versus parallelism
        2. Concurrency in Python – multithreading
      3. Thumbnail generator
        1. Thumbnail generator – producer/consumer architecture
        2. Thumbnail generator – resource constraint using locks
        3. Thumbnail generator – resource constraint using semaphores
        4. Resource constraint – semaphore versus lock
        5. Thumbnail generator – URL rate controller using conditions
      4. Multithreading – Python and GIL
        1. Concurrency in Python – multiprocessing
        2. A primality checker
        3. Sorting disk files
          1. Sorting disk files – using a counter
          2. Sorting disk files – using multiprocessing
      5. Multithreading versus multiprocessing
        1. Concurrecy in Python - Asynchronous Execution
      6. Pre-emptive versus cooperative multitasking
      7. The asyncio module in Python
      8. Waiting for a future – async and await
      9. Concurrent futures – high-level concurrent processing
        1. Disk thumbnail generator
        2. Concurrency options – how to choose?
        3. Parallel processing libraries
        4. Joblib
        5. PyMP
        6. Fractals – the Mandelbrot set
          1. Fractals – Scaling the Mandelbrot set implementation
      10. Scaling for the Web
      11. Scaling workflows – message queues and task queues
      12. Celery – a distributed task queue
        1. The Mandelbrot set using Celery
        2. Serving with Python on the Web—WSGI
        3. uWSGI – WSGI middleware on steroids
        4. Gunicorn – unicorn for WSGI
        5. Gunicorn versus uWSGI
        6. Scalability architectures
          1. Vertical scalability architectures
          2. Horizontal scalability architectures
      13. Summary
    14. 6. Security – Writing Secure Code
      1. Information Security architecture
      2. Secure coding
      3. Common security vulnerabilities
      4. Is Python secure?
        1. Reading input
        2. Evaluating arbitrary input
        3. Overflow errors
        4. Serializing objects
      5. Security issues with web applications
        1. Server Side Template Injection
        2. Server-Side Template Injection – mitigation
        3. Denial of Service
        4. Cross-Site Scripting(XSS)
        5. Mitigation – DoS and XSS
      6. Strategies for security – Python
      7. Secure coding strategies
      8. Summary
    15. 7. Design Patterns in Python
      1. Design patterns - Elements
      2. Categories of design patterns
        1. Pluggable hashing algorithms
        2. Summing up pluggable hashing algorithm
      3. Patterns in Python – Creational
        1. The Singleton pattern
          1. The Singleton – do we need a Singleton?
        2. State sharing – Borg versus Singleton
        3. The Factory pattern
        4. The Prototype pattern
          1. Prototype – deep versus shallow copy
          2. Prototype using metaclasses
          3. Combining patterns using metaclasses
          4. The Prototype factory
        5. The Builder pattern
      4. Patterns in Python – Structural
        1. The Adapter pattern
        2. The Facade pattern
          1. Facades in Python
        3. The Proxy pattern
          1. An instance-counting proxy
      5. Patterns in Python – Behavioral
        1. The Iterator pattern
        2. The Observer pattern
        3. The State pattern
      6. Summary
    16. 8. Python – Architectural Patterns
      1. Introducing MVC
        1. Model Template View (MTV) – Django
        2. Django admin – automated model-centric views
        3. Flexible Microframework – Flask
      2. Event-driven programming
        1. Chat server and client using I/O multiplexing with the select module
        2. Event-driven programming versus Concurrent programming
        3. Twisted
          1. Twisted – a simple web client
          2. Chat Server using Twisted
        4. Eventlet
        5. Greenlets and Gevent
      3. Microservice architecture
        1. Microservice frameworks in Python
        2. Microservices example – restaurant reservation
        3. Microservices – advantages
      4. Pipe and Filter architectures
        1. Pipe and Filter in Python
      5. Summary
    17. 9. Deploying Python Applications
      1. Deployability
        1. Factors affecting Deployability
      2. Tiers of software deployment architecture
      3. Software deployment in Python
        1. Packaging Python code
        2. Pip
        3. Virtualenv
        4. Virtualenv and pip
        5. Relocatable virtual environments
        6. PyPI
        7. Packaging and submission of an application
          1. The __init__.py files
          2. The setup.py file
          3. Installing the package
          4. Submitting the package to PyPI
        8. PyPA
        9. Remote deployments using Fabric
        10. Remote deployments using Ansible
        11. Managing remote daemons using Supervisor
      4. Deployment – patterns and best practices
      5. Summary
    18. 10. Techniques for Debugging
      1. Maximum subarray problem
        1. The power of "print"
        2. Analysis and rewrite
        3. Timing and optimizing the code
      2. Simple debugging tricks and techniques
        1. Word searcher program
        2. Word searcher program—debugging step 1
        3. Word searcher program—debugging step 2
        4. Word searcher program—final code
        5. Skipping blocks of code
        6. Stopping execution
        7. External dependencies—using wrappers
        8. Replacing functions with their return value/data (Mocking)
          1. Saving to / loading data from files as cache
          2. Saving to / loading data from memory as cache
          3. Returning random/mock data
            1. Generating random patient data
      3. Logging as a debugging technique
        1. Simple application logging
        2. Advanced logging—logger objects
          1. Advanced logging—custom formatting and loggers
          2. Advanced logging—writing to syslog
      4. Debugging tools—using debuggers
        1. A debugging session with pdb
        2. Pdb—similar tools
          1. iPdb
          2. Pdb++
      5. Advanced debugging—tracing
        1. The trace module
        2. The lptrace program
        3. System call tracing using strace
      6. Summary
    19. Index