book

Modern Python Cookbook - Second Edition

by Steven F. Lott

July 2020

Beginner to intermediate

822 pages

20h 28m

English

Packt Publishing

Read now

Unlock full access

What you need for this bookWho this book is forWhat this book coversTo get the most out of this bookGet in touch
Working with large and small integersChoosing between float, decimal, and fractionChoosing between true division and floor divisionRewriting an immutable stringString parsing with regular expressionsBuilding complex strings with f-stringsBuilding complicated strings from lists of charactersUsing the Unicode characters that aren't on our keyboardsEncoding strings – creating ASCII and UTF-8 bytesDecoding bytes – how to get proper characters from some bytesUsing tuples of itemsUsing NamedTuples to simplify item access in tuples
Writing Python script and module files – syntax basicsWriting long lines of codeIncluding descriptions and documentationWriting better RST markup in docstringsDesigning complex if...elif chainsSaving intermediate results with the := "walrus"Avoiding a potential problem with break statementsLeveraging exception matching rulesAvoiding a potential problem with an except: clauseConcealing an exception root causeManaging a context using the with statement
Function parameters and type hintsDesigning functions with optional parametersDesigning type hints for optional parametersUsing super flexible keyword parametersForcing keyword-only arguments with the * separatorDefining position-only parameters with the / separatorWriting hints for more complex typesPicking an order for parameters based on partial functionsWriting clear documentation strings with RST markupDesigning recursive functions around Python's stack limitsWriting testable scripts with the script-library switch
Choosing a data structureBuilding lists – literals, appending, and comprehensionsSlicing and dicing a listDeleting from a list – deleting, removing, popping, and filteringWriting list-related type hintsReversing a copy of a listBuilding sets – literals, adding, comprehensions, and operatorsRemoving items from a set – remove(), pop(), and differenceWriting set-related type hints
Creating dictionaries – inserting and updatingRemoving from dictionaries – the pop() method and the del statementControlling the order of dictionary keysWriting dictionary-related type hintsUnderstanding variables, references, and assignmentMaking shallow and deep copies of objectsAvoiding mutable default values for function parameters
Using the features of the print() functionUsing input() and getpass() for user inputDebugging with f"{value=}" stringsUsing argparse to get command-line inputUsing cmd to create command-line applicationsUsing the OS environment settings
Using a class to encapsulate data and processingEssential type hints for class definitionsDesigning classes with lots of processingUsing typing.NamedTuple for immutable objectsUsing dataclasses for mutable objectsUsing frozen dataclasses for immutable objectsOptimizing small objects with __slots__Using more sophisticated collectionsExtending a built-in collection – a list that does statisticsUsing properties for lazy attributesCreating contexts and context managersManaging multiple contexts with multiple resources
Choosing between inheritance and composition – the "is-a" questionSeparating concerns via multiple inheritanceLeveraging Python's duck typingManaging global and singleton objectsUsing more complex structures – maps of listsCreating a class that has orderable objectsImproving performance with an ordered collectionDeleting from a list of complicated objects
IntroductionWriting generator functions with the yield statementApplying transformations to a collectionUsing stacked generator expressionsPicking a subset – three ways to filterSummarizing a collection – how to reduceCombining the map and reduce transformationsImplementing "there exists" processingCreating a partial functionSimplifying complex algorithms with immutable data structuresWriting recursive generator functions with the yield from statement

Using pathlib to work with filenamesReplacing a file while preserving the previous versionReading delimited files with the CSV moduleUsing dataclasses to simplify working with CSV filesReading complex formats using regular expressionsReading JSON and YAML documentsReading XML documentsReading HTML documentsRefactoring a .csv DictReader as a dataclass reader
Test tool setupUsing docstrings for testingTesting functions that raise exceptionsHandling common doctest issuesUnit testing with the unittest moduleCombining unittest and doctest testsUnit testing with the pytest moduleCombining pytest and doctest testsTesting things that involve dates or timesTesting things that involve randomnessMocking external resources
Defining the card modelUsing the Flask framework for RESTful APIsParsing the query string in a requestMaking REST requests with urllibParsing the URL pathParsing a JSON requestImplementing authentication for web services
Finding configuration filesUsing YAML for configuration filesUsing Python for configuration filesUsing class-as-namespace for configurationDesigning scripts for compositionUsing logging for control and audit output
Combining two applications into oneCombining many applications using the Command Design PatternManaging arguments and configuration in composite applicationsWrapping and combining CLI applicationsWrapping a program and checking the outputControlling complex sequences of steps
Using the built-in statistics libraryAverage of values in a counterComputing the coefficient of a correlationComputing regression parametersComputing an autocorrelationConfirming that the data is random – the null hypothesisLocating outliersAnalyzing many variables in one pass

Content preview from Modern Python Cookbook - Second Edition

15 Statistical Programming and Linear Regression

Data analysis and statistical processing are very import applications for modern programming languages. The subject area is vast. The Python ecosystem includes a number of add-on packages that provide sophisticated data exploration, analysis, and decision-making features.

We'll look at several topics, starting with some basic statistical calculations that we can do with Python's built-in libraries and data structures. This will lead to the question of correlation and how to create a regression model.

Statistical work also raises questions of randomness and the null hypothesis. It's essential to be sure that there really is a measurable statistical effect in a set of data. We can waste a lot of ...