book

Python for Geeks

by Muhammad Asif

October 2021

Intermediate to advanced

546 pages

11h 24m

English

Packt Publishing

Read now

Unlock full access

ContributorsAbout the authorAbout the reviewers
Who this book is forWhat this book coversTo get the most out of this bookDownload the example code filesDownload the color imagesConventions usedGet in touch
Python culture and communityDifferent phases of a Python projectStrategizing the development processIterating through the phasesAiming for MVP firstStrategizing development for specialized domainsEffectively documenting Python codePython commentsDocstringFunctional or class-level documentationDeveloping an effective naming schemeMethodsVariablesConstantClassesPackagesModulesImport conventionsArgumentsUseful toolsExploring choices for source controlWhat does not belong to the source control repository?Understanding strategies for deploying the codeBatch developmentPython development environmentsIDLESublime TextPyCharmVisual Studio CodePyDevSpyderSummaryQuestionsFurther readingAnswers
Technical requirementsIntroduction to modules and packagesImporting modulesUsing the import statementUsing the __import__ statementUsing the importlib.import_module statementAbsolute versus relative importLoading and initializing a moduleStandard modulesWriting reusable modulesBuilding packagesNamingPackage initialization fileBuilding a packageAccessing packages from any locationSharing a packageBuilding a package as per the PyPA guidelinesInstalling from the local source code using pipPublishing a package to Test PyPIInstalling the package from PyPISummaryQuestionsFurther readingAnswers
Technical requirementsIntroducing classes and objectsDistinguishing between class attributes and instance attributesUsing constructors and destructors with classesDistinguishing between class methods and instance methodsSpecial methodsUnderstanding OOP principlesEncapsulation of dataEncompassing data and actionsHiding informationProtecting the dataUsing traditional getters and settersUsing property decoratorsExtending classes with inheritanceSimple inheritanceMultiple inheritancePolymorphismMethod overloadingMethod overridingAbstractionUsing composition as an alternative design approachIntroducing duck typing in PythonLearning when not to use OOP in PythonSummaryQuestionsFurther readingAnswers
Technical requirementsIntroducing Python data containersStringsListsTuplesDictionariesSetsUsing iterators and generators for data processingIteratorsGeneratorsHandling files in PythonFile operationsUsing a context managerOperating on multiple filesHandling errors and exceptionsWorking with exceptions in PythonRaising exceptionsDefining custom exceptionsUsing the Python logging moduleIntroducing core logging componentsWorking with the logging moduleWhat to log and what not to logSummaryQuestionsFurther readingAnswers
Technical requirementsUnderstanding various levels of testingUnit testingIntegration testingSystem testingAcceptance testingWorking with Python test frameworksWorking with the unittest frameworkWorking with the pytest frameworkExecuting TDDRedGreenRefactorIntroducing automated CISummaryQuestionsFurther readingAnswers
Technical requirementsLearning advanced tricks for using functions Introducing the counter, itertools, and zip functions for iterative tasksUsing filters, mappers, and reducers for data transformationsLearning how to build lambda functionsEmbedding a function within another functionModifying function behavior using decoratorsUnderstanding advanced concepts with data structuresEmbedding a dictionary inside a dictionaryUsing comprehensionIntroducing advanced tricks with pandas DataFrameLearning DataFrame operationsLearning advanced tricks for a DataFrame objectSummaryQuestionsFurther readingAnswers

Technical requirementsUnderstanding multithreading in Python and its limitationsWhat is a Python blind spot?Learning the key components of multithreaded programming in PythonCase study – a multithreaded application to download files from Google DriveGoing beyond a single CPU – implementing multiprocessingCreating multiple processes Sharing data between processesExchanging objects between processesSynchronization between processesCase study – a multiprocessor application to download files from Google DriveUsing asynchronous programming for responsive systemsUnderstanding the asyncio moduleDistributing tasks using queuesCase study – asyncio application to download files from Google DriveSummaryQuestionsFurther readingAnswers
Technical requirementsLearning about the cluster options for parallel processing Hadoop MapReduceApache SparkIntroducing RDDs Learning RDD operationsCreating RDD objectsUsing PySpark for parallel data processingCreating SparkSession and SparkContext programsExploring PySpark for RDD operationsLearning about PySpark DataFramesIntroducing PySpark SQLCase studies of using Apache Spark and PySparkCase study 1 – Pi (π) calculator on Apache SparkCase study 2 – Word cloud using PySparkSummaryQuestionsFurther readingAnswers
Technical requirementsLearning about the cloud options for Python applicationsIntroducing Python development environments for the cloudIntroducing cloud runtime options for PythonBuilding Python web services for cloud deploymentUsing Google Cloud SDKUsing the GCP web consoleUsing Google Cloud Platform for data processingLearning the fundamentals of Apache BeamIntroducing Apache Beam pipelinesBuilding pipelines for Cloud DataflowSummaryQuestionsFurther readingAnswers
Technical requirementsLearning requirements for web developmentWeb frameworksUser interfaceWeb server/application serverDatabaseSecurityAPIDocumentationIntroducing the Flask frameworkBuilding a basic application with routingHandling requests with different HTTP method typesRendering static and dynamic contentsExtracting parameters from an HTTP requestInteracting with database systemsHandling errors and exceptions in web applicationsBuilding a REST APIUsing Flask for a REST APIDeveloping a REST API for database accessCase study– Building a web application using the REST APISummaryQuestionsFurther readingAnswers
Technical requirementsIntroducing microservicesLearning best practices for microservicesBuilding microservices-based applicationsLearning microservice development options in PythonIntroducing deployment options for microservicesDeveloping a sample microservices-based applicationSummaryQuestionsFurther readingAnswers
Technical requirementsIntroducing serverless functionsBenefitsUse casesUnderstanding the deployment options for serverless functionsLearning how to build serverless functionsBuilding an HTTP-based Cloud Function using the GCP ConsoleCase study – building a notification app for cloud storage eventsSummaryQuestionsFurther readingAnswers
Technical requirementsIntroducing machine learning Using Python for machine learningIntroducing machine learning libraries in Python Best practices of training data with PythonBuilding and evaluating a machine learning modelLearning about an ML model building processBuilding a sample ML modelEvaluating a model using cross-validation and fine tuning hyperparameters Saving an ML model to a fileDeploying and predicting an ML model on GCP CloudSummaryQuestionsFurther readingAnswers
Technical requirementsIntroducing network automationMerits and challenges of network automationUse casesInteracting with network devicesProtocols for interacting with network devicesInteracting with network devices using SSH-based Python librariesInteracting with network devices using NETCONFIntegrating with network management systemsUsing location services endpointsGetting an authentication tokenGetting network devices and an interface inventoryUpdating the network device portIntegrating with event-driven systems Creating subscriptions for Apache KafkaProcessing events from Apache KafkaRenewing and deleting a subscriptionSummaryQuestionsFurther readingAnswers
Other Books You May EnjoyPackt is searching for authors like youShare Your Thoughts

Content preview from Python for Geeks

Chapter 7: Multiprocessing, Multithreading, and Asynchronous Programming

We can write efficient and optimized code for faster execution time, but there is always a limit to the amount of resources available for the processes running our programs. However, we can still improve application execution time by executing certain tasks in parallel on the same machine or across different machines. This chapter will cover parallel processing or concurrency in Python for the applications running on a single machine. We will cover parallel processing using multiple machines in the next chapter. In this chapter, we focus on the built-in support available in Python for the implementation of parallel processing. We will start with the multithreading in Python ...