book

Dead Simple Python

Name: Dead Simple Python
Author: Jason C. McDonald
ISBN: 9781718500921

by Jason C. McDonald

November 2022

Beginner to intermediate

752 pages

19h 19m

English

No Starch Press

Read now

Unlock full access

Title Page
Copyright
Dedication
About the Author
Foreword
Acknowledgments
Introduction
Who Is This Book For?What Does “Simple” Mean?What’s in This Book?What’s NOT in This BookHow to Read This BookAbout the VocabularyTheory RecapsObjective or Opinionated?The ExamplesWhat About a Project?Prerequisites
Part I: THE PYTHON ENVIRONMENT
Chapter 1: The Python Philosophy
What Is Python, Exactly?Myths: What Python Isn’tMyth #1: Python Is Merely a Scripting LanguageMyth #2: Python Is SlowMyth #3: Python Cannot Be CompiledMyth #4: Python Gets Compiled Behind the ScenesMyth #5: Python Is Unsuitable for Large ProjectsPython 2 vs. Python 3Defining “Pythonic” CodeThe Zen of PythonDocumentation, PEPs, and YouWho Calls the Shots?The Python CommunityThe Pursuit of the One Obvious WayWrapping Up
Chapter 2: Your Workbench
Installing PythonInstalling on WindowsInstalling on macOSInstalling on LinuxInstalling from SourceMeet the InterpreterInteractive SessionRunning Python FilesPackages and Virtual EnvironmentsCreating a Virtual EnvironmentActivating a Virtual EnvironmentLeaving a Virtual EnvironmentIntroducing pipSystem-Wide PackagesInstalling Packagesrequirements.txtUpgrading PackagesRemoving PackagesFinding PackagesOne Warning About pip . . .Virtual Environments and GitThe Whole ShebangFile EncodingA Few Extra Virtual Environment TricksUsing a Virtual Environment Without ActivatingThe AlternativesThe Line Limit DebateTabs or Spaces?Quality Control: Static AnalyzersPylintFlake8MypyStyle Janitors: Autoformatting Toolsautopep8BlackTesting FrameworksAn Exhibition of Code EditorsIDLEEmacs and VimPyCharmVisual Studio CodeSublime TextSpyderEclipse + PyDev/LiClipseThe Eric Python IDEWrapping Up

Chapter 3: Syntax Crash Course
Hello, World!Statements and ExpressionThe Importance of WhitespaceDoing NothingComments and DocstringsDocstringsDeclaring VariablesWhat About Constants?MathematicsMeet the Number TypesOperatorsThe math ModuleLogicConditionalsComparison OperatorsBoolean, None, and Identity OperatorsTruthinessLogical OperatorsThe Walrus OperatorEllipsisStringsString LiteralsRaw StringsFormatted StringsTemplate StringsString ConversionA Note on String ConcatenationFunctionsClasses and ObjectsError HandlingTuples and ListsLoopswhile LoopLoop Controlfor LoopStructural Pattern MatchingLiteral Patterns and WildcardsOr PatternsCapture PatternsGuard StatementsMore About Structural Pattern MatchingWrapping Up
Chapter 4: Project Structure and Imports
Setting Up the RepositoryModules and PackagesPEP 8 and NamingProject Directory StructureHow import WorksImport Dos and Don’tsImporting Functions from ModulesThe Problem of ShadowingThe Trouble with Nested PackagesBeware of Importing AllImporting Within Your ProjectAbsolute ImportsRelative ImportsImporting from the Same PackageEntry PointsModule Entry PointsPackage Entry PointsControlling Package ImportsProgram Entry PointsThe Python Module Search PathWhat Really HappensWrapping Up
Part II: ESSENTIAL STRUCTURES
Chapter 5: Variables and Types
Variables According to Python: Names and ValuesAssignmentData TypesThe type() FunctionDuck TypingScope and Garbage CollectionLocal Scope and the Reference-Counting Garbage CollectorInterpreter ShutdownGlobal ScopeThe Dangers of Global ScopeThe nonlocal KeywordScope ResolutionThe Curious Case of the ClassGenerational Garbage CollectorThe Immutable TruthPassing by AssignmentCollections and ReferencesShallow CopyDeep CopyCoercion and ConversionA Note About Systems Hungarian NotationTerminology ReviewWrapping Up
Chapter 6: Functions and Lambdas
Python Function EssentialsRecursionDefault Argument ValuesKeyword ArgumentsOn Overloaded FunctionsVariadic ArgumentsKeyword Variadic ArgumentsKeyword-Only ParametersPositional-Only ParametersArgument Types: All Together Now!Nested FunctionsClosuresRecursion with ClosuresStateful ClosuresLambdasWhy Lambdas Are UsefulLambdas as Sorting KeysDecoratorsType Hints and Function AnnotationsDuck Typing and Type HintsShould You Use Type Hinting?Wrapping Up
Chapter 7: Objects and Classes
Declaring a ClassThe InitializerThe ConstructorThe FinalizerAttributesInstance AttributesClass AttributesScope-Naming ConventionsNonpublicPublicName ManglingPublic, Nonpublic, or Name Mangled?MethodsInstance MethodsClass MethodsStatic MethodsPropertiesSetting Up the ScenarioDefining a PropertyProperty with DecoratorsWhen Not to Use PropertiesSpecial MethodsScenario SetupConversion MethodsComparison MethodsBinary Operator SupportUnary Operator SupportMaking CallableMore Special Methods: Looking AheadClass DecoratorsStructural Pattern Matching with ObjectsFunctional Meets Object OrientedWhen to Use ClassesClasses Aren’t ModulesSingle ResponsibilitySharing StateAre Objects Right for You?Wrapping Up
Chapter 8: Errors and Exceptions
Exceptions in PythonReading TracebacksCatching Exceptions: LBYL vs. EAFPMultiple ExceptionsBeware the Diaper Anti-patternRaising ExceptionsUsing ExceptionsExceptions and LoggingBubbling UpException ChainingElse and FinallyElse: “If All Goes Well”Finally: “After Everything”Creating ExceptionsA Gallery of ExceptionsWrapping Up
Part III: DATA AND FLOW
Chapter 9: Collections and Iteration
Loopswhile Loopsfor LoopsCollectionsTuplesNamed TuplesListsDequesSetsfrozensetDictionariesCheck or Except?Dictionary VariantsUnpacking CollectionsStarred ExpressionsUnpacking DictionariesStructural Pattern Matching on CollectionsAccessing by Index or KeySlice NotationStart and StopNegative IndicesStepsCopy with SliceSlice ObjectsSlicing on Custom ObjectsUsing isliceThe in OperatorChecking Collection LengthIterationIterables and IteratorsManually Using IteratorsIterating with for LoopsSorting Collections in LoopsEnumerating LoopsMutation in LoopsLoop Nesting and AlternativesIteration ToolsBasic Built-in ToolsFilterMapZipItertoolsCustom Iterable ClassesWrapping Up
Chapter 10: Generators and Comprehensions
Lazy Evaluation and Eager IterablesInfinite IteratorsGeneratorsGenerators vs. Iterator ClassesClosing GeneratorsBehavior on CloseThrowing Exceptionsyield fromGenerator ExpressionsGenerator Objects Are LazyGenerator Expressions with Multiple LoopsConditionals in Generator ExpressionsNested Generator ExpressionsList ComprehensionsSet ComprehensionsDictionary ComprehensionsHazards of Generator ExpressionsThey Quickly Become UnreadableThey Don’t Replace LoopsThey Can Be Hard to DebugWhen to Use Generator ExpressionsSimple CoroutinesReturning Values from a CoroutineSequence of BehaviorWhat About Async?Wrapping Up
Chapter 11: Text IO and Context Managers
Standard Input and OutputRevisiting print()Revisiting input()StreamsContext Manager BasicsFile ModesReading FilesThe read() MethodThe readline() MethodThe readlines() MethodReading with IterationStream PositionWriting FilesThe write() MethodThe writelines() MethodWriting Files with print()Line SeparatorsContext Manager DetailsHow Context Managers WorkUsing Multiple Context ManagersImplementing the Context Management ProtocolThe __enter__() MethodThe __exit__() MethodUsing the Custom ClassPathsPath ObjectsParts of a PathCreating a PathRelative PathsPaths Relative to PackagePath OperationsOut-of-Place File WritesThe os ModuleFile FormatsJSONOther FormatsWrapping Up
Chapter 12: Binary and Serialization
Binary Notation and BitwiseNumber Systems RefresherPython Integers and BinaryBitwise OperationsBytes LiteralsBytes-Like ObjectsCreating a bytes ObjectUsing int.to_bytes()Sequence OperationsConverting bytes to intstructstruct Format String and PackingUnpacking with structstruct objectsBitwise on Bytes-Like ObjectsBitwise Operations via IntegersBitwise Operations via IterationmemoryviewReading and Writing Binary FilesOrganizing the DataWriting to a FileReading from a Binary FileSeek with Binary StreamBufferedRWPairSerialization TechniquesForbidden Tools: pickle, marshal, and shelveSerialization FormatsWrapping Up
Part IV: ADVANCED CONCEPTS
Chapter 13: Inheritance and Mixins
When to Use InheritanceCrimes of InheritanceBasic Inheritance in PythonMultiple InheritanceMethod Resolution OrderEnsuring Consistent Method Resolution OrderExplicit Resolution OrderResolving Base Class in Multiple InheritanceMixinsWrapping Up
Chapter 14: Metaclasses and ABCs
MetaclassesCreating Classes with typeCustom MetaclassesType Expectations with Duck TypingEAFP: Catching ExceptionsLBYL: Checking for AttributesAbstract ClassesBuilt-in ABCsDeriving from ABCsImplementing Custom ABCsVirtual SubclassesSetting Up the ExampleUsing Virtual SubclassingWrapping Up
Chapter 15: Introspection and Generics
Special AttributesInside Object Attribute Access: The __dict__ Special AttributeListing AttributesGetting an AttributeChecking for an AttributeSetting an AttributeDeleting an AttributeFunction AttributesThe Wrong Way to Use Function AttributesMutability and Function AttributesDescriptorsThe Descriptor ProtocolWriting a Descriptor Class (the Slightly Wrong Way)Using a DescriptorWriting a Descriptor Class the Right WayUsing Multiple Descriptors in the Same ClassSlotsBinding Attribute Names to ValuesUsing Arbitrary Attributes with SlotsSlots and InheritanceImmutable ClassesSingle-Dispatch Generic FunctionsRegistering Single-Dispatch Functions with Type HintsRegistering Single-Dispatch Functions with Explicit TypeRegistering Single-Dispatch Functions with the register() MethodUsing the Element ClassArbitrary ExecutionWrapping Up
Chapter 16: Asynchrony and Concurrency
Asynchrony in PythonThe Example Scenario: Collatz Game, Synchronous VersionAsynchronyNative CoroutinesTasksThe Event LoopMaking It (Actually) AsynchronousScheduling and Asynchronous Execution FlowSimplifying the CodeAsynchronous IterationAsynchronous Context ManagersAsynchronous GeneratorsOther Asynchrony ConceptsWrapping Up
Chapter 17: Threading and Parallelism
ThreadingConcurrency vs. ParallelismBasic ThreadingTimeoutsDaemonic ThreadsFutures and ExecutorsTimeouts with FuturesRace ConditionsA Race Condition ExampleCreating Multiple Threads with ThreadPoolExecutorLocksDeadlock, Livelock, and StarvationPassing Messages with QueueFutures with Multiple WorkersAchieving Parallelism with MultiprocessingPickling DataSpeed Considerations and ProcessPoolExecutorThe Producer/Consumer ProblemImporting the ModulesMonitoring the QueueSubprocess CleanupConsumersChecking for an Empty QueueProducersStarting the ProcessesPerformance ResultsLogging with MultiprocessingWrapping Up
Part V: BEYOND THE CODE
Chapter 18: Packaging and Distribution
Planning Your PackagingThe Dangers of Cargo Cult ProgrammingA Note on Packaging OpinionsDetermining Your Packaging GoalsProject Structure: src or src-lessPackaging a Distribution Package with setuptoolsProject Files and StructureWhere Metadata BelongsThe README.md and LICENSE FilesThe setup.cfg FileThe setup.py FileThe MANIFEST.in FileThe requirements.txt FileThe pyproject.toml FileTesting the Setup ConfigurationBuilding Your PackagePublishing on pip (Twine)Uploading to Test PyPIInstalling Your Uploaded PackageUploading to PyPIAlternative Packaging ToolsPoetryFlitDistributing to End UsersPEXFreezersImages and ContainersA Note on Native Linux PackagingDocumentationWrapping Up
Chapter 19: Debugging and Logging
WarningsTypes of WarningsFiltering WarningsConverting Warnings into ExceptionsLoggingLogger ObjectsHandler ObjectsLogging with LevelsControlling the Log LevelRunning the ExampleFilter, Formatter, and ConfigurationAssert StatementsProper Use of assertWrong Use of assertSeeing assert in ActionThe inspect ModuleUsing pdbA Debugging ExampleStarting the DebuggerDebugger Shell CommandsStepping Through the CodeSetting a Breakpoint and Stepping into a FunctionMoving Through the Execution StackInspecting the SourceChecking a SolutionPostmortem DebuggingUsing faulthandlerEvaluating Your Program’s Security with BanditReporting Bugs to PythonWrapping Up
Chapter 20: Testing and Profiling
What About TDD?Test FrameworksThe Example ProjectTesting and Project StructureTesting BasicsStarting the ExampleUnit TestingExecuting the Tests with pytestTesting for ExceptionsTest FixturesContinuing the Example: Using the APISharing Data Between Test ModulesFlaky Tests and Conditionally Skipping TestsAdvanced Fixtures: Mocking and ParametrizingContinuing the Example: Representing a TypoParametrizingIndirect Parametrization of FixturesMocking Inputs with MonkeypatchMarkingCapturing from Standard StreamsGUI TestingContinuing the Example: Connecting the API to TypoAutouse FixturesMixed ParametrizationFuzzingWrapping Up the ExampleCode CoverageAutomating Testing with toxBenchmarking and ProfilingBenchmarking with timeitProfiling with cProfile or profiletracemallocWrapping Up
Chapter 21: The Parting of the Ways
About the FutureWhere Do You Go from Here?Application Development in PythonGame Development in PythonWeb Development in PythonClient-Side PythonData Science in PythonMachine Learning in PythonSecurityEmbedded Development in PythonScriptingPython FlavorsDeveloping for PythonDeveloping Python Packages and ToolsDeveloping Python ExtensionsContributing to PythonGetting Involved with PythonAsking QuestionsAnswering QuestionsUser GroupsPyLadiesConferencesJoining the Python Software FoundationAnd the Road Goes Ever On . . .
Appendix A: Special Attributes and Methods
Special MethodsSpecial Attributes
Appendix B: Python Debugger (pdb) Commands
Appendix : Glossary
A B C D E F G H I K L M N O P Q R S T U V W Y Z
Index

Content preview from Dead Simple Python

17 Threading and Parallelism

Before the advent of asynchrony in Python, you had two options to improve the responsiveness of your program: threading and multiprocessing. Although the two concepts are often seen as related, and even interchangeable in some languages, they couldn’t be more different in Python.

Threading is a means of achieving concurrency, which is useful in working around IO-blocking tasks, wherein the code is limited by the speed of something external, like user input, the network, or another program. It is not useful by itself for working around CPU-blocking tasks, wherein the magnitude of processing is the cause of code ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781098156671Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Dead Simple Python

by Jason C. McDonald

17 Threading and Parallelism

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.