book

Python in a Nutshell, 3rd Edition

by Alex Martelli, Anna Ravenscroft, Steve Holden

April 2017

Intermediate to advanced

769 pages

21h 56m

English

O'Reilly Media, Inc.

Read now

Unlock full access

How This Book Is OrganizedPart I, Getting Started with PythonPart II, Core Python Language and Built-insPart III, Python Library and Extension ModulesPart IV, Network and Web ProgrammingPart V, Extending, Distributing, v2/v3 MigrationConventions Used in This BookReference ConventionsVersion ConventionsTypographic ConventionsHow to Contact UsO’Reilly SafariAcknowledgments
The Python LanguageThe Python Standard Library and Extension ModulesPython ImplementationsCPythonJythonIronPythonPyPyChoosing Between CPython, Jython, IronPython, and PyPyOther Developments, Implementations, and DistributionsLicensing and Price IssuesPython Development and VersionsPython ResourcesDocumentationCommunityInstallationInstalling Python from BinariesmacOSInstalling Python from Source CodeMicrosoft WindowsUnix-Like PlatformsInstalling JythonInstalling IronPythonInstalling PyPy
The python ProgramEnvironment VariablesCommand-Line Syntax and OptionsInteractive SessionsPython Development EnvironmentsIDLEOther Python IDEsFree Text Editors with Python SupportTools for Checking Python ProgramsRunning Python ProgramsThe jython InterpreterThe IronPython InterpreterThe PyPy Interpreter
Lexical StructureLines and IndentationCharacter SetsTokensStatementsData TypesNumbersSequencesSetsDictionariesNoneCallablesBoolean ValuesVariables and Other ReferencesVariablesAssignment Statementsdel StatementsExpressions and OperatorsComparison ChainingShort-Circuiting OperatorsNumeric OperationsNumeric ConversionsArithmetic OperationsComparisonsBitwise Operations on IntegersSequence OperationsSequences in GeneralStringsTuplesListsSet OperationsSet MembershipSet MethodsDictionary OperationsDictionary MembershipIndexing a DictionaryDictionary MethodsControl Flow StatementsThe if StatementThe while StatementThe for StatementThe break StatementThe continue StatementThe else Clause on Loop StatementsThe pass StatementThe try and raise StatementsThe with StatementFunctionsThe def StatementParameters“Keyword-only” Parameters (v3 Only)Attributes of Function ObjectsFunction Annotations and Type Hints (v3 Only)The return StatementCalling FunctionsNamespaceslambda ExpressionsGeneratorsRecursion
Classes and InstancesPython ClassesThe class StatementThe Class BodyDescriptorsInstancesAttribute Reference BasicsBound and Unbound MethodsInheritanceThe Built-in object TypeClass-Level MethodsProperties__slots____getattribute__Per-Instance MethodsInheritance from Built-in TypesSpecial MethodsGeneral-Purpose Special MethodsSpecial Methods for ContainersAbstract Base ClassesSpecial Methods for Numeric ObjectsDecoratorsMetaclassesHow Python v2 Determines a Class’s MetaclassHow Python v3 Determines a Class’s MetaclassHow a Metaclass Creates a Class
The try Statementtry/excepttry/finallyThe try/except/finally StatementThe with Statement and Context ManagersGenerators and ExceptionsException PropagationThe raise StatementException ObjectsThe Hierarchy of Standard ExceptionsStandard Exception ClassesCustom Exception ClassesCustom Exceptions and Multiple InheritanceOther Exceptions Used in the Standard LibraryError-Checking StrategiesLBYL Versus EAFPHandling Errors in Large ProgramsLogging ErrorsThe assert StatementThe __debug__ Built-in Variable
Module ObjectsThe import StatementThe from StatementModule LoadingBuilt-in ModulesSearching the Filesystem for a ModuleThe Main ProgramReloading ModulesCircular Importssys.modules EntriesCustom ImportersPackagesSpecial Attributes of Package ObjectsNamespace Packages (v3 Only)Absolute Versus Relative ImportsDistribution Utilities (distutils) and setuptoolsPython Wheels (and Eggs)Python EnvironmentsEnter the Virtual EnvironmentWhat Is a Virtual Environment?Creating and Deleting Virtual EnvironmentsWorking with Virtual EnvironmentsManaging Dependency RequirementsBest Practices with virtualenvs
Built-in TypesBuilt-in FunctionsThe sys ModuleThe copy ModuleThe collections ModuleChainMap (v3 Only)CounterOrderedDictdefaultdictdequenamedtupleThe functools ModuleThe heapq ModuleThe Decorate-Sort-Undecorate IdiomThe argparse ModuleThe itertools Module
Methods of String and Bytes ObjectsThe string ModuleString FormattingValue SelectionValue ConversionValue FormattingNew in 3.6: Formatted String LiteralsLegacy String Formatting with %Format Specifier SyntaxText Wrapping and FillingThe pprint ModuleThe reprlib ModuleUnicodeThe codecs ModuleThe unicodedata Module
Regular Expressions and the re Modulere and bytes Versus Unicode StringsPattern-String SyntaxCommon Regular Expression IdiomsSets of CharactersAlternativesGroupsOptional FlagsMatch Versus SearchAnchoring at String Start and EndRegular Expression ObjectsMatch ObjectsFunctions of the re Module

Other Chapters That Also Deal with FilesOrganization of This ChapterThe io ModuleCreating a “file” Object with io.openAttributes and Methods of “file” ObjectsIteration on “File” ObjectsFile-Like Objects and PolymorphismThe tempfile ModuleAuxiliary Modules for File I/OThe fileinput ModuleThe linecache ModuleThe struct ModuleIn-Memory “Files”: io.StringIO and io.BytesIOCompressed FilesThe gzip ModuleThe bz2 ModuleThe tarfile ModuleThe zipfile ModuleThe zlib ModuleThe os ModuleOSError ExceptionsThe errno ModuleFilesystem OperationsPath-String Attributes of the os ModulePermissionsFile and Directory Functions of the os ModuleThe os.path ModuleThe stat ModuleThe filecmp ModuleThe fnmatch ModuleThe glob ModuleThe shutil ModuleFile Descriptor OperationsText Input and OutputStandard Output and Standard ErrorThe print FunctionStandard InputThe getpass ModuleRicher-Text I/OThe readline ModuleConsole I/OInteractive Command SessionsInitializing a Cmd InstanceMethods of Cmd InstancesAttributes of Cmd InstancesA Cmd ExampleInternationalizationThe locale ModuleThe gettext ModuleMore Internationalization Resources
SerializationThe json ModuleThe pickle and cPickle ModulesThe shelve ModuleDBM ModulesThe v3 dbm Packagev2’s dbm modulesExamples of DBM-Like File UseBerkeley DB InterfacingThe Python Database API (DBAPI) 2.0Exception ClassesThread SafetyParameter StyleFactory FunctionsType Description AttributesThe connect FunctionConnection ObjectsCursor ObjectsDBAPI-Compliant ModulesSQLiteclass sqlite3.Connectionclass sqlite3.Row
The time ModuleThe datetime ModuleThe date ClassThe time ClassThe datetime ClassThe timedelta ClassThe pytz ModuleThe dateutil ModuleThe sched ModuleThe calendar Module
Site and User CustomizationThe site and sitecustomize ModulesUser CustomizationTermination FunctionsDynamic Execution and execAvoiding execExpressionsCompile and Code ObjectsNever exec or eval Untrusted CodeInternal TypesType ObjectsThe Code Object TypeThe frame TypeGarbage CollectionThe gc ModuleThe weakref Module
Threads in PythonThe threading ModuleThread ObjectsThread Synchronization ObjectsThread Local StorageThe queue ModuleMethods of Queue InstancesThe multiprocessing ModuleDifferences Between Multiprocessing and ThreadingSharing State: Classes Value, Array, and ManagerMultiprocessing PoolThe concurrent.futures ModuleThreaded Program ArchitectureProcess EnvironmentRunning Other ProgramsRunning Other Programs with the os ModuleThe Subprocess ModuleThe mmap ModuleMethods of mmap ObjectsUsing mmap Objects for IPC
The math and cmath ModulesThe operator ModuleRandom and Pseudorandom NumbersPhysically Random and Cryptographically Strong Random NumbersThe random ModuleThe fractions ModuleThe decimal ModuleThe gmpy2 ModuleArray ProcessingThe array ModuleExtensions for Numeric Array ComputationNumPyCreating a NumPy ArrayShape, Indexing, and SlicingMatrix Operations in NumPySciPy
TestingUnit Testing and System TestingThe doctest ModuleThe unittest ModuleDebuggingBefore You DebugThe inspect ModuleThe traceback ModuleThe pdb ModuleThe warnings ModuleClassesObjectsFiltersFunctionsOptimizationDeveloping a Fast-Enough Python ApplicationBenchmarkingLarge-Scale OptimizationProfilingSmall-Scale Optimization
Networking PrinciplesThe Berkeley Socket InterfaceSocket AddressesClient-Server ComputingThe socket ModuleSocket ObjectsA Connectionless Socket ClientA Connectionless Socket ServerA Connection-Oriented Socket ClientA Connection-Oriented Socket ServerTransport Layer Security (TLS, AKA SSL)SSLContext
Coroutine-Based Async ArchitecturesThe asyncio Module (v3 Only)Coroutines in asyncioasyncio’s Event LoopSynchronization and QueuesThe selectors Moduleselectors EventsThe SelectorKey ClassThe BaseSelector ClassWhen to Use selectors
Email ProtocolsThe poplib ModuleThe smtplib ModuleHTTP and URL ClientsURL AccessThe Third-Party requests PackageThe urllib Package (v3)The urllib Module (v2)The urllib2 Module (v2)Other Network Protocols
WSGIWSGI ServersPython Web FrameworksFull-Stack Versus Lightweight FrameworksA Few Popular Full-Stack FrameworksSome Popular Lightweight Frameworks
MIME and Email Format HandlingFunctions in the email PackageThe email.message ModuleThe email.Generator ModuleCreating MessagesThe email.encoders ModuleThe email.utils ModuleExample Uses of the email Packagerfc822 and mimetools Modules (v2)Encoding Binary Data as ASCII TextThe base64 ModuleThe quopri ModuleThe uu Module
The html.entities (v2: htmlentitydefs) ModuleThe BeautifulSoup Third-Party PackageThe BeautifulSoup ClassThe Navigable Classes of bs4bs4 find... Methods (“Search Methods”)bs4 CSS SelectorsAn HTML Parsing Example with BeautifulSoupGenerating HTMLEditing and Creating HTML with bs4Building HTML with bs4TemplatingThe jinja2 PackageThe jinja2.Environment ClassThe jinja2.Template Class
ElementTreeThe Element ClassThe ElementTree ClassFunctions in the ElementTree ModuleParsing XML with ElementTree.parseBuilding an ElementTree from ScratchParsing XML Iteratively
Extending Python with Python’s C APIBuilding and Installing C-Coded Python ExtensionsOverview of C-Coded Python Extension ModulesReturn Values of Python’s C API FunctionsThe Initialization ModuleReference CountingAccessing ArgumentsCreating Python ValuesExceptionsAbstract Layer FunctionsConcrete Layer FunctionsA Simple Extension ExampleDefining New TypesExtending Python Without Python’s C APIctypesCythonThe cdef and cpdef Statements and Function ParametersThe ctypedef StatementThe for…from StatementCython ExpressionsA Cython Example: Greatest Common DivisorEmbedding PythonInstalling Resident Extension ModulesSetting ArgumentsPython Initialization and FinalizationRunning Python Code
setuptoolsThe Distribution and Its RootThe setup.py ScriptThe requirements.txt FileThe MANIFEST.in FileThe setup.cfg FileDistributing Your PackageCreate the DistributionRegistering and Uploading to a Repository
Preparing for Python 3Minimizing Syntax DifferencesAvoid “Old-Style” Classesprint as a FunctionString LiteralsNumeric ConstantsText and Binary DataNever Sort Using cmpexcept ClausesDivisionIncompatible Syntax to Avoid for CompatibilityChoosing Your Support StrategySteady As She Goes—v2-only Supportv2/v3 Support with Conversion2to3 Conversion: A Case Studyv2/v3 Support with a Single Source Treev3-only Support

Content preview from Python in a Nutshell, 3rd Edition

Chapter 26. v2/v3 Migration and Coexistence

The earliest release of Python 3 (the precursor of what we call v3 in this book) first appeared in 2008, specifically as “Python 3.0”: that release was not a production-quality one, nor was it meant to be. Its main purpose was to let people start adapting to changes in syntax and semantics; some parts of the implementation were unsatisfactory (particularly the I/O facilities, now much improved). As we’re writing this chapter, the current releases are 2.7.12 and 3.5.2, with a 3.6.0 release just out in December 2016, very close to the deadline for us to finish this book (as a result, while we mention 3.6’s highlights as being “new in 3.6,” that’s in good part based on a beta version of 3.6: we can’t claim thorough coverage of 3.6).

Python’s core developers have done a lot of work to backport v3 features into v2, but this activity has now ended, and v2 is feature-frozen (some v3 standard library backports to v2 are available for pip install, as repeatedly mentioned earlier in this book). v2 is getting only security bug-fix releases, and even those are due to stop once v2 maintenance formally ends in 2020. Guido van Rossum’s PyCon NA keynote in 2014 explicitly ruled out any 2.8 release; should you choose to stick with v2, after 2020 you will be on your own.

For those who have mastered Python’s build system, or who are able to hire third-party developers to support their v2 use, “sticking with v2” may remain a viable strategy even after 2020. ...