Hands-On Software Engineering with Python

Hands-On Software Engineering with Python

by Brian Allbee
Released October 2018
Publisher(s): Packt Publishing
ISBN: 9781788622011

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Explore various verticals in software engineering through high-end systems using Python

Key Features

  • Master the tools and techniques used in software engineering
  • Evaluates available database options and selects one for the final Central Office system-components
  • Experience the iterations software go through and craft enterprise-grade systems

Book Description

Software Engineering is about more than just writing code—it includes a host of soft skills that apply to almost any development effort, no matter what the language, development methodology, or scope of the project. Being a senior developer all but requires awareness of how those skills, along with their expected technical counterparts, mesh together through a project's life cycle. This book walks you through that discovery by going over the entire life cycle of a multi-tier system and its related software projects. You'll see what happens before any development takes place, and what impact the decisions and designs made at each step have on the development process. The development of the entire project, over the course of several iterations based on real-world Agile iterations, will be executed, sometimes starting from nothing, in one of the fastest growing languages in the world—Python. Application of practices in Python will be laid out, along with a number of Python-specific capabilities that are often overlooked. Finally, the book will implement a high-performance computing solution, from first principles through complete foundation.

What you will learn

  • Understand what happens over the course of a system's life (SDLC)
  • Establish what to expect from the pre-development life cycle steps
  • Find out how the development-specific phases of the SDLC affect development
  • Uncover what a real-world development process might be like, in an Agile way
  • Find out how to do more than just write the code
  • Identify the existence of project-independent best practices and how to use them
  • Find out how to design and implement a high-performance computing process

Who this book is for

Hands-On Software Engineering with Python is for you if you are a developer having basic understanding of programming and its paradigms and want to skill up as a senior programmer. It is assumed that you have basic Python knowledge.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. Hands-On Software Engineering with Python
  3. Humble Bundle
  4. Packt Upsell
    1. Why subscribe?
    2. Packt.com
  5. Contributors
    1. About the author
    2. About the reviewers
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  7. Programming versus Software Engineering
    1. The bigger picture
    2. Asking questions
    3. Summary
  8. The Software Development Life Cycle
    1. Pre-development phases of the SDLC
      1. Initial concept/vision
      2. Concept development
      3. Project management planning
    2. Development – specific phases of the SDLC
      1. Requirements analysis and definition
      2. System architecture and design
      3. Development and quality assurance
      4. System integration, testing, and acceptance
    3. Post-development phases of the SDLC
    4. Summary
  9. System Modeling
    1. Architecture, both logical and physical
      1. Logical architecture
      2. Physical architecture
    2. Use cases (business processes and rules)
    3. Data structure and flow
    4. Interprocess communication
    5. System scope and scale
    6. Summary
  10. Methodologies, Paradigms, and Practices
    1. Process methodologies
      1. Waterfall
      2. Agile (in general)
      3. Scrum
        1. Scrum and the phases of the SDLC model
      4. Kanban
        1. Kanban and the phases of the SDLC model
      5. Other Agile methodologies
        1. Extreme programming
        2. Feature-driven development
        3. Test-driven design
    2. Development paradigms
      1. Object-oriented programming
      2. Functional programming
    3. Development practices
      1. Continuous integration
      2. Continuous delivery or deployment
    4. Summary
  11. The hms_sys System Project
    1. Goals for the system
    2. What's known/designed before development starts
    3. What the iteration chapters will look like
      1. Iteration goals and stories
      2. Writing and testing the code
      3. Post-development considerations and impact
    4. Summary
  12. Development Tools and Best Practices
    1. Development tools
      1. Integrated Development Environment (IDE) options
        1. IDLE
        2. Geany
        3. Eclipse variations + PyDev
        4. Others
      2. Source Code Management
        1. Typical SCM activities
        2. Git
        3. Subversion
        4. Basic workflows for Git and SVN compared
        5. Other SCM options
    2. Best practices
      1. Standards for code
        1. PEP-8
        2. Internal standards
          1. Code organization in modules
          2. Structure and standards for classes
          3. Function and method annotation (hinting)
      2. Process standards
        1. Unit testing
        2. Repeatable build processes
        3. Integrating unit tests and build processes
      3. Defining package structures for Python code
        1. Packages in a project's context
      4. Using Python virtual environments
    3. Summary
  13. Setting Up Projects and Processes
    1. Iteration goals
    2. Assembly of stories and tasks
    3. Setting Up SCM
    4. Stubbing out component projects
      1. Component project analysis
      2. Component project setup
        1. Packaging and build process
        2. Python virtual environments
      3. Basic unit testing
      4. Identifying missing test case classes
      5. Identifying missing test methods
      6. Creating reusable module code coverage tests
      7. The property and method testing decorators
      8. Creating unit test template files
    5. Integrating tests with the build process
    6. Summary
  14. Creating Business Objects
    1. Iteration goals
    2. Assembly of stories and tasks
    3. A quick review of classes
    4. Implementing the basic business objects in hms_sys
      1. Address
      2. BaseArtisan
        1. OO principles – composition over inheritance
        2. Implementing BaseArtisan's properties
        3. Implementing BaseArtisan's methods
      3. BaseCustomer
      4. BaseOrder
      5. BaseProduct
      6. Dealing with duplicated code – HasProducts
    5. Summary
  15. Testing Business Objects
    1. Starting the unit testing process
      1. Unit testing the Address class
      2. Unit testing HasProducts
      3. Unit testing BaseProduct
      4. Unit testing BaseOrder
      5. Unit-testing BaseCustomer
      6. Unit testing BaseArtisan
    2. Unit testing patterns established so far
    3. Distribution and installation considerations
    4. Quality assurance and acceptance
    5. Operation/use, maintenance, and decommissioning considerations
    6. Summary
  16. Thinking About Business Object Data Persistence
    1. Iterations are (somewhat) flexible
    2. Data storage options
      1. Relational databases
        1. Advantages and drawbacks
        2. MySQL/MariaDB
        3. MS-SQL
        4. PostgresQL
      2. NoSQL databases
        1. Advantages and drawbacks
        2. MongoDB
        3. Other NoSQL options
      3. Other data storage options
    3. Selecting a data storage option
    4. Polymorphism (and programming to an interface)
    5. Data access design strategies
      1. Data access decisions
      2. Why start from scratch?
    6. Summary
  17. Data Persistence and BaseDataObject
    1. The BaseDataObject ABC
    2. Unit testing BaseDataObject
    3. Summary
  18. Persisting Object Data to Files
    1. Setting up the hms_artisan project
    2. Creating a local file system data store
    3. Implementing JSONFileDataObject
    4. The concrete business objects of hms_artisan
      1. Dealing with is_dirty and properties
      2. hms_artisan.Artisan
      3. hms_artisan.Product
      4. hms_artisan.Order
    5. Summary
  19. Persisting Data to a Database
    1. The Artisan Gateway and Central Office application objects
      1. Picking out a backend datastore engine
      2. The data access strategy for the Central Office projects
      3. Supporting objects for data persistence
      4. RDBMS implementations
    2. The concrete business objects of the Central Office projects
      1. hms_core.co_objects.Artisan
      2. hms_core.co_objects.Product
      3. Other hms_core.co_objects classes
      4. Accounting for the other CRUD operations
    3. Summary
  20. Testing Data Persistence
    1. Writing the unit tests
    2. Testing hms_artisan.data_storage
      1. Testing hms_artisan.artisan_objects
    3. Testing the new hms_core Classes
      1. Unit testing hms_core.data_storage.py
      2. Unit testing hms_core.co_objects.py
    4. Unit tests and trust
    5. Building/distribution, demonstration, and acceptance
    6. Operations/use, maintenance, and decommissioning considerations
    7. Summary
  21. Anatomy of a Service
    1. What is a service?
    2. Service structure
      1. Configuration
        1. Windows-style .ini files
        2. JSON files
        3. YAML files
      2. Logging service activities
      3. Handling requests and generating responses
        1. Filesystem – based
        2. HTTP- or web-based
        3. Message- queue-based
        4. Other request types
        5. Request and response formats
    3. A generic service design
      1. The BaseDaemon ABC
      2. The BaseRequestHandler and BaseResponseFormatter ABCs
    4. Integrating a service with the OS
      1. Running a service using systemctl (Linux)
      2. Running a service using NSSM (Windows)
      3. macOS, launchd, and launchctl
      4. Managing services on other systems
    5. Summary
  22. The Artisan Gateway Service
    1. Overview and goal
    2. Iteration stories
    3. Messages
    4. Deciding on a message-transmission mechanism
      1. Message-queue implementation with RabbitMQ
      2. Handling messages
      3. Queues and related Artisan properties
      4. Requirements for a web-service-based daemon
    5. Traffic to and from the service
    6. Impacts on testing and deployment
    7. Summary
  23. Handling Service Transactions
    1. Remaining stories
    2. A bit of reorganization
    3. Preparation for object transactions
    4. Product object transactions
      1. Artisan – creating a product
      2. Central Office – approving/listing a product
      3. Central Office – altering product data
      4. Artisan – updating product data
      5. Artisan – deleting a product
    5. Artisan object transactions
      1. Central Office – creating an artisan
      2. Central Office – updating artisan data
      3. Central Office – deleting an artisan
      4. Artisan – updating Artisan data
    6. Order object transactions
      1. Customer – relaying order items to artisans
      2. Customer – canceling an order
      3. Artisan – fulfilling an item in an order
    7. When do messages get sent?
    8. Summary
  24. Testing and Deploying Services
    1. The challenges of testing services
    2. The overall testing strategy
      1. Unit testing variations of note
      2. Testing Artisan transactions
    3. Demonstrating the service
    4. Packaging and deploying the service
      1. Common considerations across all operating systems
      2. Linux (systemd) execution
      3. Windows (NSSM) execution
    5. Where hms_sys development could go from here
      1. Code review, refactoring, and cleanup
      2. Developing a UI
      3. Order fulfilment and shipping APIs
    6. Summary
  25. Multiprocessing and HPC in Python
    1. Common factors to consider
    2. A simple but expensive algorithm
      1. Some testing setup
    3. Local parallel processing
      1. Threads
    4. Parallelizing across multiple machines
      1. Common functionality
      2. The Worker nodes
      3. The Orchestrator
      4. The Dispatcher
    5. Integrating Python with large-scale, cluster computing frameworks
      1. Python, Hadoop, and Spark
    6. Summary 
  26. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: Hands-On Software Engineering with Python
  • Author(s): Brian Allbee
  • Release date: October 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781788622011