book

Introducing Python, 2nd Edition

by Bill Lubanovic

November 2019

Beginner

630 pages

12h 20m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Includes

Includes Quizzes

Has Sandbox

AudienceChanges in the Second EditionOutlinePython VersionsConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
MysteriesLittle ProgramsA Bigger ProgramPython in the Real WorldPython Versus the Language from Planet XWhy Python?Why Not Python?Python 2 Versus Python 3Installing PythonRunning PythonUsing the Interactive InterpreterUsing Python FilesWhat’s Next?Your Moment of ZenComing UpThings to Do
Python Data Are ObjectsTypesMutabilityLiteral ValuesVariablesAssignmentVariables Are Names, Not PlacesAssigning to Multiple NamesReassigning a NameCopyingChoose Good Variable NamesComing UpThings to Do
BooleansIntegersLiteral IntegersInteger OperationsIntegers and VariablesPrecedenceBasesType ConversionsHow Big Is an int?FloatsMath FunctionsComing UpThings to Do
Comment with #Continue Lines with \Compare with if, elif, and elseWhat Is True?Do Multiple Comparisons with inNew: I Am the WalrusComing UpThings to Do
Create with QuotesCreate with str()Escape with \Combine by Using +Duplicate with *Get a Character with []Get a Substring with a SliceGet Length with len()Split with split()Combine by Using join()Substitute by Using replace()Strip with strip()Search and SelectCaseAlignmentFormattingOld style: %New style: {} and format()Newest Style: f-stringsMore String ThingsComing UpThings to Do
Repeat with whileCancel with breakSkip Ahead with continueCheck break Use with elseIterate with for and inCancel with breakSkip with continueCheck break Use with elseGenerate Number Sequences with range()Other IteratorsComing UpThings to Do
TuplesCreate with Commas and ()Create with tuple()Combine Tuples by Using +Duplicate Items with *Compare TuplesIterate with for and inModify a TupleListsCreate with []Create or Convert with list()Create from a String with split()Get an Item by [ offset ]Get Items with a SliceAdd an Item to the End with append()Add an Item by Offset with insert()Duplicate All Items with *Combine Lists by Using extend() or +Change an Item by [ offset ]Change Items with a SliceDelete an Item by Offset with delDelete an Item by Value with remove()Get an Item by Offset and Delete It with pop()Delete All Items with clear()Find an Item’s Offset by Value with index()Test for a Value with inCount Occurrences of a Value with count()Convert a List to a String with join()Reorder Items with sort() or sorted()Get Length with len()Assign with =Copy with copy(), list(), or a SliceCopy Everything with deepcopy()Compare ListsIterate with for and inIterate Multiple Sequences with zip()Create a List with a ComprehensionLists of ListsTuples Versus ListsThere Are No Tuple ComprehensionsComing UpThings to Do
DictionariesCreate with {}Create with dict()Convert with dict()Add or Change an Item by [ key ]Get an Item by [key] or with get()Get All Keys with keys()Get All Values with values()Get All Key-Value Pairs with items()Get Length with len()Combine Dictionaries with {**a, **b}Combine Dictionaries with update()Delete an Item by Key with delGet an Item by Key and Delete It with pop()Delete All Items with clear()Test for a Key with inAssign with =Copy with copy()Copy Everything with deepcopy()Compare DictionariesIterate with for and inDictionary ComprehensionsSetsCreate with set()Convert with set()Get Length with len()Add an Item with add()Delete an Item with remove()Iterate with for and inTest for a Value with inCombinations and OperatorsSet ComprehensionsCreate an Immutable Set with frozenset()Data Structures So FarMake Bigger Data StructuresComing UpThings to Do

Define a Function with defCall a Function with ParenthesesArguments and ParametersNone Is UsefulPositional ArgumentsKeyword ArgumentsSpecify Default Parameter ValuesExplode/Gather Positional Arguments with *Explode/Gather Keyword Arguments with **Keyword-Only ArgumentsMutable and Immutable ArgumentsDocstringsFunctions Are First-Class CitizensInner FunctionsClosuresAnonymous Functions: lambdaGeneratorsGenerator FunctionsGenerator ComprehensionsDecoratorsNamespaces and ScopeUses of _ and __ in NamesRecursionAsync FunctionsExceptionsHandle Errors with try and exceptMake Your Own ExceptionsComing UpThings to Do
What Are Objects?Simple ObjectsDefine a Class with classAttributesMethodsInitializationInheritanceInherit from a Parent ClassOverride a MethodAdd a MethodGet Help from Your Parent with super()Multiple InheritanceMixinsIn self DefenseAttribute AccessDirect AccessGetters and SettersProperties for Attribute AccessProperties for Computed ValuesName Mangling for PrivacyClass and Object AttributesMethod TypesInstance MethodsClass MethodsStatic MethodsDuck TypingMagic MethodsAggregation and CompositionWhen to Use Objects or Something ElseNamed TuplesDataclassesAttrsComing UpThings to Do
Modules and the import StatementImport a ModuleImport a Module with Another NameImport Only What You Want from a ModulePackagesThe Module Search PathRelative and Absolute ImportsNamespace PackagesModules Versus ObjectsGoodies in the Python Standard LibraryHandle Missing Keys with setdefault() and defaultdict()Count Items with Counter()Order by Key with OrderedDict()Stack + Queue == dequeIterate over Code Structures with itertoolsPrint Nicely with pprint()Get RandomMore Batteries: Get Other Python CodeComing UpThings to Do
Text Strings: UnicodePython 3 Unicode StringsUTF-8EncodeDecodeHTML EntitiesNormalizationFor More InformationText Strings: Regular ExpressionsFind Exact Beginning Match with match()Find First Match with search()Find All Matches with findall()Split at Matches with split()Replace at Matches with sub()Patterns: Special CharactersPatterns: Using SpecifiersPatterns: Specifying match() OutputBinary Databytes and bytearrayConvert Binary Data with structOther Binary Data ToolsConvert Bytes/Strings with binascii()Bit OperatorsA Jewelry AnalogyComing UpThings to Do
Leap YearThe datetime ModuleUsing the time ModuleRead and Write Dates and TimesAll the ConversionsAlternative ModulesComing UpThings to Do
File Input and OutputCreate or Open with open()Write a Text File with print()Write a Text File with write()Read a Text File with read(), readline(), or readlines()Write a Binary File with write()Read a Binary File with read()Close Files Automatically by Using withChange Position with seek()Memory MappingFile OperationsCheck Existence with exists()Check Type with isfile()Copy with copy()Change Name with rename()Link with link() or symlink()Change Permissions with chmod()Change Ownership with chown()Delete a File with remove()Directory OperationsCreate with mkdir()Delete with rmdir()List Contents with listdir()Change Current Directory with chdir()List Matching Files with glob()PathnamesGet a Pathname with abspath()Get a symlink Pathname with realpath()Build a Pathname with os.path.join()Use pathlibBytesIO and StringIOComing UpThings to Do
Programs and ProcessesCreate a Process with subprocessCreate a Process with multiprocessingKill a Process with terminate()Get System Info with osGet Process Info with psutilCommand AutomationInvokeOther Command HelpersConcurrencyQueuesProcessesThreadsconcurrent.futuresGreen Threads and geventtwistedasyncioRedisBeyond QueuesComing UpThings to Do
Flat Text FilesPadded Text FilesTabular Text FilesCSVXMLAn XML Security NoteHTMLJSONYAMLTablibPandasConfiguration FilesBinary FilesPadded Binary Files and Memory MappingSpreadsheetsHDF5TileDBRelational DatabasesSQLDB-APISQLiteMySQLPostgreSQLSQLAlchemyOther Database Access PackagesNoSQL Data StoresThe dbm FamilyMemcachedRedisDocument DatabasesTime Series DatabasesGraph DatabasesOther NoSQLFull-Text DatabasesComing UpThings to Do
TCP/IPSocketsScapyNetcatNetworking PatternsThe Request-Reply PatternZeroMQOther Messaging ToolsThe Publish-Subscribe PatternRedisZeroMQOther Pub-Sub ToolsInternet ServicesDomain Name SystemPython Email ModulesOther ProtocolsWeb Services and APIsData SerializationSerialize with pickleOther Serialization FormatsRemote Procedure CallsXML RPCJSON RPCMessagePack RPCZerorpcgRPCTwirpRemote Management ToolsBig Fat DataHadoopSparkDiscoDaskCloudsAmazon Web ServicesGoogle CloudMicrosoft AzureOpenStackDockerKubernetesComing UpThings to Do
Web ClientsTest with telnetTest with curlTest with httpieTest with httpbinPython’s Standard Web LibrariesBeyond the Standard Library: requestsWeb ServersThe Simplest Python Web ServerWeb Server Gateway Interface (WSGI)ASGIApacheNGINXOther Python Web ServersWeb Server FrameworksBottleFlaskDjangoOther FrameworksDatabase FrameworksWeb Services and AutomationwebbrowserwebviewWeb APIs and RESTCrawl and ScrapeScrapyBeautifulSoupRequests-HTMLLet’s Watch a MovieComing UpThings to Do
About ProgrammingFind Python CodeInstall PackagesUse pipUse virtualenvUse pipenvUse a Package ManagerInstall from SourceIntegrated Development EnvironmentsIDLEPyCharmIPythonJupyter NotebookJupyterLabName and DocumentAdd Type HintsTestCheck with pylint, pyflakes, flake8, or pep8Test with unittestTest with doctestTest with noseOther Test FrameworksContinuous IntegrationDebug Python CodeUse print()Use DecoratorsUse pdbUse breakpoint()Log Error MessagesOptimizeMeasure TimingAlgorithms and Data StructuresCython, NumPy, and C ExtensionsPyPyNumbaSource ControlMercurialGitDistribute Your ProgramsClone This BookHow You Can Learn MoreBooksWebsitesGroupsConferencesGetting a Python JobComing UpThings to Do
2-D GraphicsStandard LibraryPIL and PillowImageMagick3-D Graphics3-D AnimationGraphical User InterfacesPlots, Graphs, and VisualizationMatplotlibSeabornBokehGamesAudio and MusicComing UpThings to Do
The Microsoft Office SuiteCarrying Out Business TasksProcessing Business DataExtracting, Transforming, and LoadingData ValidationAdditional Sources of InformationOpen Source Python Business PackagesPython in FinanceBusiness Data SecurityMapsFormatsDraw a Map from a ShapefileGeopandasOther Mapping PackagesApplications and DataComing UpThings to Do
Math and Statistics in the Standard LibraryMath FunctionsWorking with Complex NumbersCalculate Accurate Floating Point with decimalPerform Rational Arithmetic with fractionsUse Packed Sequences with arrayHandling Simple Stats with statisticsMatrix MultiplicationScientific PythonNumPyMake an Array with array()Make an Array with arange()Make an Array with zeros(), ones(), or random()Change an Array’s Shape with reshape()Get an Element with []Array MathLinear AlgebraSciPySciKitPandasPython and Scientific AreasComing UpThings to Do
HardwareCaveman ComputersElectricityInventionsAn Idealized ComputerThe CPUMemory and CachesStorageInputsOutputsRelative Access TimesSoftwareIn the Beginning Was the BitMachine LanguageAssemblerHigher-Level LanguagesOperating SystemsVirtual MachinesContainersDistributed Computing and NetworksThe CloudKubernetes
Check Your Python VersionInstall Standard PythonmacOSWindowsLinux or UnixInstall the pip Package ManagerInstall virtualenvOther Packaging SolutionsInstall AnacondaInstall Anaconda’s Package Manager conda
Coroutines and Event LoopsAsyncio AlternativesAsync Versus…Async Frameworks and Servers
1. A Taste of Py2. Data: Types, Values, Variables, and Names3. Numbers4. Choose with if5. Text Strings6. Loop with while and for7. Tuples and Lists8. Dictionaries9. Functions10. Oh Oh: Objects and Classes11. Modules, Packages, and Goodies12. Wrangle and Mangle Data13. Calendars and Clocks14. Files and Directories15. Data in Time: Processes and Concurrency16. Data in a Box: Persistent Storage17. Data in Space: Networks18. The Web, Untangled19. Be a Pythonista20. Py Art21. Py at Work22. PySci
Operator PrecedenceString MethodsChange CaseSearchModifyFormatString TypeString Module AttributesCoda

Content preview from Introducing Python, 2nd Edition

Chapter 15. Data in Time: Processes and Concurrency

One thing a computer can do that most humans can’t is be sealed up in a cardboard box and sit in a warehouse.

Jack Handey

This chapter and the next two are a bit more challenging than earlier ones. In this one we cover data in time (sequential and concurrent access on a single computer), and following that we look at data in a box (storage and retrieval with special files and databases) in Chapter 16 and then data in space (networking) in Chapter 17.

Programs and Processes

When you run an individual program, your operating system creates a single process. It uses system resources (CPU, memory, disk space) and data structures in the operating system’s kernel (file and network connections, usage statistics, and so on). A process is isolated from other processes—it can’t see what other processes are doing or interfere with them.

The operating system keeps track of all the running processes, giving each a little time to run and then switching to another, with the twin goals of spreading the work around fairly and being responsive to the user. You can see the state of your processes with graphical interfaces such as the Mac’s Activity Monitor (macOS), Task Manager on Windows-based computers, or the top command in Linux.

You can also access process data from your own programs. The standard library’s os module provides a common way of accessing some system information. For instance, the following functions get the process ID and ...