book

Software Architecture with Python

by Anand Balachandran Pillai

April 2017

Intermediate to advanced

556 pages

11h 5m

English

Packt Publishing

Read now

Unlock full access

What this book covers

Downloading the example codeDownloading the color images of this bookErrataPiracyQuestions
Defining software architectureSoftware architecture versus designAspects of software architecture
An architecture defines a structureAn architecture picks a core set of elementsAn architecture captures early design decisionsAn architecture manages stakeholder requirementsAn architecture influences the organizational structureAn architecture is influenced by its environmentAn architecture documents the systemAn architecture often conforms to a pattern
ModifiabilityTestabilityScalabilityPerformanceAvailabilitySecurityDeployability
What is modifiability?
Python and readabilityReadability – antipatternsTechniques for readabilityDocument your codeFollow coding and style guidelinesReview and refactor codeCommenting the code
Measuring cohesion and couplingMeasuring cohesion and coupling – string and text processing
Providing explicit interfacesReducing two-way dependenciesAbstract common servicesUsing inheritance techniquesUsing late binding techniques
What are code smells?Cyclomatic complexity – the McCabe metricTesting for metricsRunning static checkers
Refactoring code – fixing complexityRefactoring code – fixing code smellsRefactoring code – fixing styling and coding issues
Understanding testabilitySoftware testability and related attributesTestability – architectural aspectsTestability – strategiesReduce system complexityImproving predictabilityControl and isolate external dependencies
Unit testingUnit testing in actionExtending our unit test caseNosing around with nose2Testing with py.testCode coverageMeasuring coverage using coverage.pyMeasuring coverage using nose2Measuring coverage using pytestMocking things upTests inline in documentation – doctestsIntegration testsTest automationTest automation using Selenium WebDriverTest-driven developmentTDD with palindromes
What is performance?
Measuring time using a context managerTiming code using the timeit moduleMeasuring the performance of our code using timeitFinding out time complexity – graphsMeasuring CPU time with timeit
Deterministic profilingProfiling with cProfile and profilePrime number iterator class – performance tweaksProfiling – collecting and reporting statisticsThird-party profilersLine profilerMemory profilerSubstring (subsequence) problem
objgraphPympler
Mutable containers – lists, dictionaries, and setsListsDictionariesSetsImmutable containers – tuplesHigh performance containers – the collections moduledequedefaultdictOrderedDictDropping duplicates from a container without losing the orderImplementing a Least Recently Used (LRU) cache dictionaryCounterChainMapnamedtupleProbabilistic data structures – bloom filters
Scalability and performance
Concurrency versus parallelismConcurrency in Python – multithreading
Thumbnail generator – producer/consumer architectureThumbnail generator – resource constraint using locksThumbnail generator – resource constraint using semaphoresResource constraint – semaphore versus lockThumbnail generator – URL rate controller using conditions
Concurrency in Python – multiprocessingA primality checkerSorting disk filesSorting disk files – using a counterSorting disk files – using multiprocessing
Concurrecy in Python – Asynchronous Execution
Disk thumbnail generatorConcurrency options – how to choose?Parallel processing librariesJoblibPyMPFractals – the Mandelbrot setFractals – scaling the Mandelbrot set implementation
The Mandelbrot set using CeleryServing with Python on the Web – WSGIuWSGI – WSGI middleware on steroidsGunicorn – unicorn for WSGIGunicorn versus uWSGIScalability architecturesVertical scalability architecturesHorizontal scalability architectures
Information security architecture
Reading inputEvaluating arbitrary inputOverflow errorsSerializing objects
Server Side Template InjectionServer-Side Template Injection – MitigationDenial of ServiceCross-Site Scripting (XSS)Mitigation – DoS and XSS
Design patterns – elements
Pluggable hashing algorithmsSumming up pluggable hashing algorithm
The Singleton patternThe Singleton – do we need a Singleton?State sharing – Borg versus SingletonThe Factory patternThe Prototype patternPrototype – deep versus shallow copyPrototype using metaclassesCombining patterns using metaclassesThe Prototype factoryThe Builder pattern
The Adapter patternThe Facade patternFacades in PythonThe proxy patternAn instance-counting proxy
The Iterator patternThe Observer patternThe State pattern
Introducing MVCModel Template View (MTV) – DjangoDjango admin – automated model-centric viewsFlexible Microframework – Flask
Chat server and client using I/O multiplexing with the select moduleEvent-driven programming versus concurrent programmingTwistedTwisted – a simple web clientChat server using TwistedEventletGreenlets and Gevent
Microservice frameworks in PythonMicroservices example – restaurant reservationMicroservices – advantages
Pipe and filter in Python
DeployabilityFactors affecting deployability
Packaging Python codePIPVirtualenvVirtualenv and pipRelocatable virtual environmentsPyPIPackaging and submission of an applicationThe __init__.py filesThe setup.py fileInstalling the packageSubmitting the package to PyPIPyPARemote deployments using FabricRemote deployments using AnsibleManaging remote daemons using Supervisor
Maximum subarray problemThe power of "print"Analysis and rewriteTiming and optimizing the code
Word searcher programWord searcher program – debugging step 1Word searcher program – debugging step 2Word searcher program – final codeSkipping blocks of codeStopping executionExternal dependencies – using wrappersReplacing functions with their return value/data (mocking)Saving to / loading data from files as cacheSaving to / loading data from memory as cacheReturning random/mock dataGenerating random patient data
Simple application loggingAdvanced logging – logger objectsAdvanced logging – custom formatting and loggersAdvanced logging – writing to syslog
A debugging session with pdbPdb – similar toolsiPdbPdb++
The trace moduleThe lptrace programSystem call tracing using strace

Content preview from Software Architecture with Python

Multithreading versus multiprocessing

Now that we have come to the end of our discussion on multiprocessing, it is a good time to compare and contrast the scenarios where one needs to choose between scaling using threads in a single process or using multiple processes in Python.

Here are some guidelines.

Use multithreading in the following cases:

The program needs to maintain a lot of shared states, especially mutable ones. A lot of the standard data structures in Python, such as lists, dictionaries, and others, are thread-safe, so it costs much less to maintain a mutable shared state using threads than via processes.
The program needs to keep a low memory foot-print.
The program spends a lot of time doing I/O. Since the GIL is released by threads ...