book

Python in a Nutshell, 3rd Edition

by Alex Martelli, Anna Ravenscroft, Steve Holden

April 2017

Intermediate to advanced

769 pages

21h 56m

English

O'Reilly Media, Inc.

Read now

Unlock full access

Preface
How This Book Is OrganizedPart I, Getting Started with PythonPart II, Core Python Language and Built-insPart III, Python Library and Extension ModulesPart IV, Network and Web ProgrammingPart V, Extending, Distributing, v2/v3 MigrationConventions Used in This BookReference ConventionsVersion ConventionsTypographic ConventionsHow to Contact UsO’Reilly SafariAcknowledgments
1. Introduction to Python
The Python LanguageThe Python Standard Library and Extension ModulesPython ImplementationsCPythonJythonIronPythonPyPyChoosing Between CPython, Jython, IronPython, and PyPyOther Developments, Implementations, and DistributionsLicensing and Price IssuesPython Development and VersionsPython ResourcesDocumentationCommunityInstallationInstalling Python from BinariesmacOSInstalling Python from Source CodeMicrosoft WindowsUnix-Like PlatformsInstalling JythonInstalling IronPythonInstalling PyPy
2. The Python Interpreter
The python ProgramEnvironment VariablesCommand-Line Syntax and OptionsInteractive SessionsPython Development EnvironmentsIDLEOther Python IDEsFree Text Editors with Python SupportTools for Checking Python ProgramsRunning Python ProgramsThe jython InterpreterThe IronPython InterpreterThe PyPy Interpreter
3. The Python Language
Lexical StructureLines and IndentationCharacter SetsTokensStatementsData TypesNumbersSequencesSetsDictionariesNoneCallablesBoolean ValuesVariables and Other ReferencesVariablesAssignment Statementsdel StatementsExpressions and OperatorsComparison ChainingShort-Circuiting OperatorsNumeric OperationsNumeric ConversionsArithmetic OperationsComparisonsBitwise Operations on IntegersSequence OperationsSequences in GeneralStringsTuplesListsSet OperationsSet MembershipSet MethodsDictionary OperationsDictionary MembershipIndexing a DictionaryDictionary MethodsControl Flow StatementsThe if StatementThe while StatementThe for StatementThe break StatementThe continue StatementThe else Clause on Loop StatementsThe pass StatementThe try and raise StatementsThe with StatementFunctionsThe def StatementParameters“Keyword-only” Parameters (v3 Only)Attributes of Function ObjectsFunction Annotations and Type Hints (v3 Only)The return StatementCalling FunctionsNamespaceslambda ExpressionsGeneratorsRecursion
4. Object-Oriented Python
Classes and InstancesPython ClassesThe class StatementThe Class BodyDescriptorsInstancesAttribute Reference BasicsBound and Unbound MethodsInheritanceThe Built-in object TypeClass-Level MethodsProperties__slots____getattribute__Per-Instance MethodsInheritance from Built-in TypesSpecial MethodsGeneral-Purpose Special MethodsSpecial Methods for ContainersAbstract Base ClassesSpecial Methods for Numeric ObjectsDecoratorsMetaclassesHow Python v2 Determines a Class’s MetaclassHow Python v3 Determines a Class’s MetaclassHow a Metaclass Creates a Class
5. Exceptions
The try Statementtry/excepttry/finallyThe try/except/finally StatementThe with Statement and Context ManagersGenerators and ExceptionsException PropagationThe raise StatementException ObjectsThe Hierarchy of Standard ExceptionsStandard Exception ClassesCustom Exception ClassesCustom Exceptions and Multiple InheritanceOther Exceptions Used in the Standard LibraryError-Checking StrategiesLBYL Versus EAFPHandling Errors in Large ProgramsLogging ErrorsThe assert StatementThe __debug__ Built-in Variable
6. Modules
Module ObjectsThe import StatementThe from StatementModule LoadingBuilt-in ModulesSearching the Filesystem for a ModuleThe Main ProgramReloading ModulesCircular Importssys.modules EntriesCustom ImportersPackagesSpecial Attributes of Package ObjectsNamespace Packages (v3 Only)Absolute Versus Relative ImportsDistribution Utilities (distutils) and setuptoolsPython Wheels (and Eggs)Python EnvironmentsEnter the Virtual EnvironmentWhat Is a Virtual Environment?Creating and Deleting Virtual EnvironmentsWorking with Virtual EnvironmentsManaging Dependency RequirementsBest Practices with virtualenvs
7. Core Built-ins and Standard Library Modules
Built-in TypesBuilt-in FunctionsThe sys ModuleThe copy ModuleThe collections ModuleChainMap (v3 Only)CounterOrderedDictdefaultdictdequenamedtupleThe functools ModuleThe heapq ModuleThe Decorate-Sort-Undecorate IdiomThe argparse ModuleThe itertools Module
8. Strings and Things
Methods of String and Bytes ObjectsThe string ModuleString FormattingValue SelectionValue ConversionValue FormattingNew in 3.6: Formatted String LiteralsLegacy String Formatting with %Format Specifier SyntaxText Wrapping and FillingThe pprint ModuleThe reprlib ModuleUnicodeThe codecs ModuleThe unicodedata Module
9. Regular Expressions
Regular Expressions and the re Modulere and bytes Versus Unicode StringsPattern-String SyntaxCommon Regular Expression IdiomsSets of CharactersAlternativesGroupsOptional FlagsMatch Versus SearchAnchoring at String Start and EndRegular Expression ObjectsMatch ObjectsFunctions of the re Module

10. File and Text Operations
Other Chapters That Also Deal with FilesOrganization of This ChapterThe io ModuleCreating a “file” Object with io.openAttributes and Methods of “file” ObjectsIteration on “File” ObjectsFile-Like Objects and PolymorphismThe tempfile ModuleAuxiliary Modules for File I/OThe fileinput ModuleThe linecache ModuleThe struct ModuleIn-Memory “Files”: io.StringIO and io.BytesIOCompressed FilesThe gzip ModuleThe bz2 ModuleThe tarfile ModuleThe zipfile ModuleThe zlib ModuleThe os ModuleOSError ExceptionsThe errno ModuleFilesystem OperationsPath-String Attributes of the os ModulePermissionsFile and Directory Functions of the os ModuleThe os.path ModuleThe stat ModuleThe filecmp ModuleThe fnmatch ModuleThe glob ModuleThe shutil ModuleFile Descriptor OperationsText Input and OutputStandard Output and Standard ErrorThe print FunctionStandard InputThe getpass ModuleRicher-Text I/OThe readline ModuleConsole I/OInteractive Command SessionsInitializing a Cmd InstanceMethods of Cmd InstancesAttributes of Cmd InstancesA Cmd ExampleInternationalizationThe locale ModuleThe gettext ModuleMore Internationalization Resources
11. Persistence and Databases
SerializationThe json ModuleThe pickle and cPickle ModulesThe shelve ModuleDBM ModulesThe v3 dbm Packagev2’s dbm modulesExamples of DBM-Like File UseBerkeley DB InterfacingThe Python Database API (DBAPI) 2.0Exception ClassesThread SafetyParameter StyleFactory FunctionsType Description AttributesThe connect FunctionConnection ObjectsCursor ObjectsDBAPI-Compliant ModulesSQLiteclass sqlite3.Connectionclass sqlite3.Row
12. Time Operations
The time ModuleThe datetime ModuleThe date ClassThe time ClassThe datetime ClassThe timedelta ClassThe pytz ModuleThe dateutil ModuleThe sched ModuleThe calendar Module
13. Controlling Execution
Site and User CustomizationThe site and sitecustomize ModulesUser CustomizationTermination FunctionsDynamic Execution and execAvoiding execExpressionsCompile and Code ObjectsNever exec or eval Untrusted CodeInternal TypesType ObjectsThe Code Object TypeThe frame TypeGarbage CollectionThe gc ModuleThe weakref Module
14. Threads and Processes
Threads in PythonThe threading ModuleThread ObjectsThread Synchronization ObjectsThread Local StorageThe queue ModuleMethods of Queue InstancesThe multiprocessing ModuleDifferences Between Multiprocessing and ThreadingSharing State: Classes Value, Array, and ManagerMultiprocessing PoolThe concurrent.futures ModuleThreaded Program ArchitectureProcess EnvironmentRunning Other ProgramsRunning Other Programs with the os ModuleThe Subprocess ModuleThe mmap ModuleMethods of mmap ObjectsUsing mmap Objects for IPC
15. Numeric Processing
The math and cmath ModulesThe operator ModuleRandom and Pseudorandom NumbersPhysically Random and Cryptographically Strong Random NumbersThe random ModuleThe fractions ModuleThe decimal ModuleThe gmpy2 ModuleArray ProcessingThe array ModuleExtensions for Numeric Array ComputationNumPyCreating a NumPy ArrayShape, Indexing, and SlicingMatrix Operations in NumPySciPy
16. Testing, Debugging, and Optimizing
TestingUnit Testing and System TestingThe doctest ModuleThe unittest ModuleDebuggingBefore You DebugThe inspect ModuleThe traceback ModuleThe pdb ModuleThe warnings ModuleClassesObjectsFiltersFunctionsOptimizationDeveloping a Fast-Enough Python ApplicationBenchmarkingLarge-Scale OptimizationProfilingSmall-Scale Optimization
17. Networking Basics
Networking PrinciplesThe Berkeley Socket InterfaceSocket AddressesClient-Server ComputingThe socket ModuleSocket ObjectsA Connectionless Socket ClientA Connectionless Socket ServerA Connection-Oriented Socket ClientA Connection-Oriented Socket ServerTransport Layer Security (TLS, AKA SSL)SSLContext
18. Asynchronous Alternatives
Coroutine-Based Async ArchitecturesThe asyncio Module (v3 Only)Coroutines in asyncioasyncio’s Event LoopSynchronization and QueuesThe selectors Moduleselectors EventsThe SelectorKey ClassThe BaseSelector ClassWhen to Use selectors
19. Client-Side Network Protocol Modules
Email ProtocolsThe poplib ModuleThe smtplib ModuleHTTP and URL ClientsURL AccessThe Third-Party requests PackageThe urllib Package (v3)The urllib Module (v2)The urllib2 Module (v2)Other Network Protocols
20. Serving HTTP
WSGIWSGI ServersPython Web FrameworksFull-Stack Versus Lightweight FrameworksA Few Popular Full-Stack FrameworksSome Popular Lightweight Frameworks
21. Email, MIME, and Other Network Encodings
MIME and Email Format HandlingFunctions in the email PackageThe email.message ModuleThe email.Generator ModuleCreating MessagesThe email.encoders ModuleThe email.utils ModuleExample Uses of the email Packagerfc822 and mimetools Modules (v2)Encoding Binary Data as ASCII TextThe base64 ModuleThe quopri ModuleThe uu Module
22. Structured Text: HTML
The html.entities (v2: htmlentitydefs) ModuleThe BeautifulSoup Third-Party PackageThe BeautifulSoup ClassThe Navigable Classes of bs4bs4 find... Methods (“Search Methods”)bs4 CSS SelectorsAn HTML Parsing Example with BeautifulSoupGenerating HTMLEditing and Creating HTML with bs4Building HTML with bs4TemplatingThe jinja2 PackageThe jinja2.Environment ClassThe jinja2.Template Class
23. Structured Text: XML
ElementTreeThe Element ClassThe ElementTree ClassFunctions in the ElementTree ModuleParsing XML with ElementTree.parseBuilding an ElementTree from ScratchParsing XML Iteratively
24. Extending and Embedding Classic Python
Extending Python with Python’s C APIBuilding and Installing C-Coded Python ExtensionsOverview of C-Coded Python Extension ModulesReturn Values of Python’s C API FunctionsThe Initialization ModuleReference CountingAccessing ArgumentsCreating Python ValuesExceptionsAbstract Layer FunctionsConcrete Layer FunctionsA Simple Extension ExampleDefining New TypesExtending Python Without Python’s C APIctypesCythonThe cdef and cpdef Statements and Function ParametersThe ctypedef StatementThe for…from StatementCython ExpressionsA Cython Example: Greatest Common DivisorEmbedding PythonInstalling Resident Extension ModulesSetting ArgumentsPython Initialization and FinalizationRunning Python Code
25. Distributing Extensions and Programs
setuptoolsThe Distribution and Its RootThe setup.py ScriptThe requirements.txt FileThe MANIFEST.in FileThe setup.cfg FileDistributing Your PackageCreate the DistributionRegistering and Uploading to a Repository
26. v2/v3 Migration and Coexistence
Preparing for Python 3Minimizing Syntax DifferencesAvoid “Old-Style” Classesprint as a FunctionString LiteralsNumeric ConstantsText and Binary DataNever Sort Using cmpexcept ClausesDivisionIncompatible Syntax to Avoid for CompatibilityChoosing Your Support StrategySteady As She Goes—v2-only Supportv2/v3 Support with Conversion2to3 Conversion: A Case Studyv2/v3 Support with a Single Source Treev3-only Support
Index

Content preview from Python in a Nutshell, 3rd Edition

Chapter 3. The Python Language

This chapter is a guide to the Python language. To learn Python from scratch, we suggest you start with the appropriate links from https://www.python.org/about/gettingstarted/, depending on whether you’re a programming beginner or already have some programming experience. If you already know other programming languages well, and just want to learn specifics about Python, this chapter is for you. However, we’re not trying to teach Python: we cover a lot of ground at a pretty fast pace. We focus on the rules, and only secondarily point out best practices and style; as your Python style guide, use PEP 8 (optionally augmented by extra guidelines such as The Hitchhiker’s Guide’s, CKAN’s, and/or Google’s).

Lexical Structure

The lexical structure of a programming language is the set of basic rules that govern how you write programs in that language. It is the lowest-level syntax of the language, specifying such things as what variable names look like and how to denote comments. Each Python source file, like any other text file, is a sequence of characters. You can also usefully consider it a sequence of lines, tokens, or statements. These different lexical views complement each other. Python is very particular about program layout, especially regarding lines and indentation: pay attention to this information if you are coming to Python from another language.

Lines and Indentation

A Python program is a sequence of logical lines, each made up of one or ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781491913833Errata Page

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design