book

Learning Python, 3rd Edition

by Mark Lutz

October 2007

Beginner

752 pages

19h 44m

English

O'Reilly Media, Inc.

Read now

Unlock full access

About This Third EditionThis Edition’s Python Language ChangesThis Edition’s Python Training ChangesThis Edition’s Structural ChangesThis Edition’s Scope ChangesAbout This BookThis Book’s PrerequisitesThis Book’s Scope and Other BooksThis Book’s Style and StructureBook UpdatesAbout the Programs in This BookPreparing for Python 3.0About This SeriesUsing Code ExamplesFont ConventionsSafari® Books OnlineHow to Contact UsAcknowledgments
Why Do People Use Python?Software QualityDeveloper ProductivityIs Python a “Scripting Language”?OK, but What’s the Downside?Who Uses Python Today?What Can I Do with Python?Systems ProgrammingGUIsInternet ScriptingComponent IntegrationDatabase ProgrammingRapid PrototypingNumeric and Scientific ProgrammingGaming, Images, AI, XML, Robots, and MoreWhat Are Python’s Technical Strengths?It’s Object OrientedIt’s FreeIt’s PortableIt’s PowerfulIt’s MixableIt’s Easy to UseIt’s Easy to LearnIt’s Named After Monty PythonHow Does Python Stack Up to Language X?Chapter Summary
Introducing the Python InterpreterProgram ExecutionThe Programmer’s ViewPython’s ViewByte code compilationThe Python Virtual Machine (PVM)Performance implicationsDevelopment implicationsExecution Model VariationsPython Implementation AlternativesCPythonJythonIronPythonExecution Optimization ToolsThe Psyco just-in-time compilerThe Shedskin C++ translatorFrozen BinariesFuture Possibilities?Chapter Summary
Interactive CodingUsing the Interactive PromptSystem Command Lines and FilesUsing Command Lines and FilesUnix Executable Scripts (#!)Clicking File IconsClicking Icons on WindowsThe raw_input TrickOther Icon-Click LimitationsModule Imports and ReloadsThe Grander Module Story: AttributesModules and namespacesimport and reload Usage NotesThe IDLE User InterfaceIDLE BasicsUsing IDLEAdvanced IDLE ToolsOther IDEsEmbedding CallsFrozen Binary ExecutablesText Editor Launch OptionsOther Launch OptionsFuture Possibilities?Which Option Should I Use?Chapter SummaryBRAIN BUILDERPart I Exercises
Why Use Built-in Types?Python’s Core Data TypesNumbersStringsSequence OperationsImmutabilityType-Specific MethodsGetting HelpOther Ways to Code StringsPattern MatchingListsSequence OperationsType-Specific OperationsBounds CheckingNestingList ComprehensionsDictionariesMapping OperationsNesting RevisitedSorting Keys: for LoopsIteration and OptimizationMissing Keys: if TestsTuplesWhy Tuples?FilesOther File-Like ToolsOther Core TypesHow to Break Your Code’s FlexibilityUser-Defined ClassesAnd Everything ElseChapter Summary
Python Numeric TypesNumeric LiteralsBuilt-in Numeric Tools and ExtensionsPython Expression OperatorsMixed Operators Follow Operator PrecedenceParentheses Group SubexpressionsMixed Types Are Converted UpPreview: Operator OverloadingNumbers in ActionVariables and Basic ExpressionsNumeric Display FormatsDivision: Classic, Floor, and TrueBitwise OperationsLong IntegersComplex NumbersHexadecimal and Octal NotationOther Built-in Numeric ToolsOther Numeric TypesDecimal NumbersSetsBooleansThird-Party ExtensionsChapter Summary

The Case of the Missing Declaration StatementsVariables, Objects, and ReferencesTypes Live with Objects, Not VariablesObjects Are Garbage-CollectedShared ReferencesShared References and In-Place ChangesShared References and EqualityDynamic Typing Is EverywhereChapter Summary
String LiteralsSingle- and Double-Quoted Strings Are the SameEscape Sequences Represent Special BytesRaw Strings Suppress EscapesTriple Quotes Code Multiline Block StringsUnicode Strings Encode Larger Character SetsStrings in ActionBasic OperationsIndexing and SlicingExtended slicing: the third limitString Conversion ToolsCharacter code conversionsChanging StringsString FormattingAdvanced String FormattingDictionary-Based String FormattingString MethodsString Method Examples: Changing StringsString Method Examples: Parsing TextOther Common String Methods in ActionThe Original string ModuleGeneral Type CategoriesTypes Share Operation Sets by CategoriesMutable Types Can Be Changed In-PlaceChapter Summary
ListsLists in ActionBasic List OperationsIndexing, Slicing, and MatrixesChanging Lists In-PlaceIndex and slice assignmentsList method callsOther common list operationsDictionariesDictionaries in ActionBasic Dictionary OperationsChanging Dictionaries In-PlaceMore Dictionary MethodsA Languages TableDictionary Usage NotesUsing dictionaries to simulate flexible listsUsing dictionaries for sparse data structuresAvoiding missing-key errorsUsing dictionaries as “records”Other ways to make dictionariesChapter Summary
TuplesTuples in ActionTuple syntax peculiarities: commas and parenthesesConversions and immutabilityWhy Lists and Tuples?FilesOpening FilesUsing FilesFiles in ActionStoring and parsing Python objects in filesStoring native Python objects with pickleStoring and parsing packed binary data in filesOther File ToolsType Categories RevisitedObject FlexibilityReferences Versus CopiesComparisons, Equality, and TruthThe Meaning of True and False in PythonPython’s Type HierarchiesOther Types in PythonBuilt-in Type GotchasAssignment Creates References, Not CopiesRepetition Adds One Level DeepBeware of Cyclic Data StructuresImmutable Types Can’t Be Changed In-PlaceChapter SummaryBRAIN BUILDERPart II Exercises
Python Program Structure RevisitedPython’s StatementsA Tale of Two ifsWhat Python AddsWhat Python RemovesParentheses are optionalEnd of line is end of statementEnd of indentation is end of blockWhy Indentation Syntax?A Few Special CasesStatement rule special casesBlock rule special caseA Quick Example: Interactive LoopsA Simple Interactive LoopDoing Math on User InputsHandling Errors by Testing InputsHandling Errors with try StatementsNesting Code Three Levels DeepChapter Summary
Assignment StatementsAssignment Statement FormsSequence AssignmentsAdvanced sequence assignment patternsMultiple-Target AssignmentsMultiple-target assignment and shared referencesAugmented AssignmentsAugmented assignment and shared referencesVariable Name RulesNaming conventionsNames have no type, but objects doExpression StatementsExpression Statements and In-Place Changesprint StatementsThe Python “Hello World” ProgramRedirecting the Output StreamThe print >> file ExtensionChapter Summary
if StatementsGeneral FormatBasic ExamplesMultiway BranchingPython Syntax RulesBlock DelimitersStatement DelimitersA Few Special CasesTruth TestsThe if/else Ternary ExpressionChapter Summary
while LoopsGeneral FormatExamplesbreak, continue, pass, and the Loop elseGeneral Loop FormatExamplespasscontinuebreakelseMore on the loop else clausefor LoopsGeneral FormatExamplesBasic usageOther data typesTuple assignment in forNested for loopsIterators: A First LookFile IteratorsOther Built-in Type IteratorsOther Iteration ContextsUser-Defined IteratorsLoop Coding TechniquesCounter Loops: while and rangeNonexhaustive Traversals: rangeChanging Lists: rangeParallel Traversals: zip and mapDictionary construction with zipGenerating Both Offsets and Items: enumerateList Comprehensions: A First LookList Comprehension BasicsUsing List Comprehensions on FilesExtended List Comprehension SyntaxChapter Summary
Python Documentation Sources# CommentsThe dir FunctionDocstrings: _ _doc_ _User-defined docstringsDocstring standardsBuilt-in docstringsPyDoc: The help FunctionPyDoc: HTML ReportsStandard Manual SetWeb ResourcesPublished BooksCommon Coding GotchasChapter SummaryBRAIN BUILDERPart III Exercises
Why Use Functions?Coding Functionsdef Statementsdef Executes at RuntimeA First Example: Definitions and CallsDefinitionCallsPolymorphism in PythonA Second Example: Intersecting SequencesDefinitionCallsPolymorphism RevisitedLocal VariablesChapter Summary
Scope RulesPython Scope BasicsName Resolution: The LEGB RuleScope ExampleThe Built-in ScopeThe global StatementMinimize Global VariablesMinimize Cross-File ChangesOther Ways to Access GlobalsScopes and Nested FunctionsNested Scope DetailsNested Scope ExamplesFactory functionsRetaining enclosing scopes’ state with defaultsNested scopes and lambdasScopes versus defaults with loop variablesArbitrary scope nestingPassing ArgumentsArguments and Shared ReferencesAvoiding Mutable Argument ChangesSimulating Output ParametersSpecial Argument-Matching ModesKeyword and Default ExamplesKeywordsDefaultsArbitrary Arguments ExamplesCollecting argumentsUnpacking argumentsCombining Keywords and DefaultsThe min Wakeup CallFull creditBonus pointsThe punch lineA More Useful Example: General Set FunctionsArgument Matching: The Gritty DetailsChapter Summary
Anonymous Functions: lambdalambda ExpressionsWhy Use lambda?How (Not) to Obfuscate Your Python CodeNested lambdas and ScopesApplying Functions to ArgumentsThe apply Built-inPassing keyword argumentsapply-Like Call SyntaxMapping Functions over Sequences: mapFunctional Programming Tools: filter and reduceList Comprehensions Revisited: MappingsList Comprehension BasicsAdding Tests and Nested LoopsList Comprehensions and MatrixesComprehending List ComprehensionsIterators Revisited: GeneratorsGenerator Function ExampleExtended Generator Function Protocol: send Versus nextIterators and Built-in TypesGenerator Expressions: Iterators Meet List ComprehensionsTiming Iteration AlternativesFunction Design ConceptsFunctions Are Objects: Indirect CallsFunction GotchasLocal Names Are Detected StaticallyDefaults and Mutable ObjectsFunctions Without returnsEnclosing Scope Loop VariablesChapter SummaryBRAIN BUILDERPart IV Exercises
Why Use Modules?Python Program ArchitectureHow to Structure a ProgramImports and AttributesStandard Library ModulesHow Imports Work1. Find ItThe module search pathThe sys.path listModule file selectionAdvanced module selection concepts2. Compile It (Maybe)3. Run ItChapter Summary
Module CreationModule UsageThe import StatementThe from statementThe from * StatementImports Happen Only Onceimport and from Are AssignmentsCross-File Name Changesimport and from EquivalencePotential Pitfalls of the from StatementWhen import is requiredModule NamespacesFiles Generate NamespacesAttribute Name QualificationImports Versus ScopesNamespace NestingReloading Modulesreload Basicsreload ExampleChapter Summary
Package Import BasicsPackages and Search Path SettingsPackage _ _init_ _.py FilesPackage Import Examplefrom Versus import with PackagesWhy Use Package Imports?A Tale of Three SystemsChapter Summary
Data Hiding in ModulesMinimizing from * Damage: _X and _ _all_ _Enabling Future Language FeaturesMixed Usage Modes: _ _name_ _ and _ _main_ _Unit Tests with _ _name_ _Changing the Module Search PathThe import as ExtensionRelative Import SyntaxWhy Relative Imports?Module Design ConceptsModules Are Objects: MetaprogramsModule GotchasStatement Order Matters in Top-Level CodeImporting Modules by Name Stringfrom Copies Names but Doesn’t Linkfrom * Can Obscure the Meaning of Variablesreload May Not Impact from Importsreload, from, and Interactive Testingreload Isn’t Applied TransitivelyRecursive from Imports May Not WorkChapter SummaryBRAIN BUILDERPart V Exercises
Why Use Classes?OOP from 30,000 FeetAttribute Inheritance SearchClasses and InstancesClass Method CallsCoding Class TreesOOP Is About Code ReuseChapter Summary
Classes Generate Multiple Instance ObjectsClass Objects Provide Default BehaviorInstance Objects Are Concrete ItemsA First ExampleClasses Are Customized by InheritanceA Second ExampleClasses Are Attributes in ModulesClasses Can Intercept Python OperatorsA Third ExampleWhy Use Operator Overloading?The World’s Simplest Python ClassChapter Summary
The class StatementGeneral FormExampleMethodsExampleCalling Superclass ConstructorsOther Method Call PossibilitiesInheritanceAttribute Tree ConstructionSpecializing Inherited MethodsClass Interface TechniquesAbstract SuperclassesOperator OverloadingCommon Operator Overloading Methods_ _getitem_ _ Intercepts Index References_ _getitem_ _ and _ _iter_ _ Implement IterationUser-Defined IteratorsMultiple iterators on one object_ _getattr_ _ and _ _setattr_ _ Catch Attribute ReferencesEmulating Privacy for Instance Attributes_ _repr_ _ and _ _str_ _ Return String Representations_ _radd_ _ Handles Right-Side Addition_ _call_ _ Intercepts CallsFunction Interfaces and Callback-Based Code_ _del_ _ Is a DestructorNamespaces: The Whole StorySimple Names: Global Unless AssignedAttribute Names: Object NamespacesThe “Zen” of Python Namespaces: Assignments Classify NamesNamespace DictionariesNamespace LinksA More Realistic ExampleChapter Summary
Python and OOPOverloading by Call Signatures (or Not)Classes As RecordsOOP and Inheritance: “Is-a” RelationshipsOOP and Composition: “Has-a” RelationshipsStream Processors RevisitedOOP and DelegationMultiple InheritanceClasses Are Objects: Generic Object FactoriesWhy Factories?Methods Are Objects: Bound or UnboundDocumentation Strings RevisitedClasses Versus ModulesChapter Summary
Extending Built-in TypesExtending Types by EmbeddingExtending Types by SubclassingPseudoprivate Class AttributesName Mangling OverviewWhy Use Pseudoprivate Attributes?New-Style ClassesDiamond Inheritance ChangeDiamond inheritance exampleExplicit conflict resolutionOther New-Style Class ExtensionsStatic and class methodsInstance slotsClass propertiesNew _ _getattribute_ _ overloading methodStatic and Class MethodsUsing Static and Class MethodsFunction DecoratorsDecorator ExampleClass GotchasChanging Class Attributes Can Have Side EffectsMultiple Inheritance: Order MattersMethods, Classes, and Nested Scopes“Overwrapping-itis”Chapter SummaryBRAIN BUILDERPart VI Exercises
Why Use Exceptions?Exception RolesException Handling: The Short StoryThe try/except/else Statementtry Statement ClausesThe try/else ClauseExample: Default BehaviorExample: Catching Built-in ExceptionsThe try/finally StatementExample: Coding Termination Actions with try/finallyUnified try/except/finallyCombining finally and except by NestingUnified try ExampleThe raise StatementExample: Raising and Catching User-Defined ExceptionsExample: Passing Extra Data with raiseExample: Propagating Exceptions with raiseThe assert StatementExample: Trapping Constraints (but Not Errors)with/as Context ManagersBasic UsageThe Context Management ProtocolChapter Summary
String-Based ExceptionsString Exceptions Are Right Out!Class-Based ExceptionsClass Exception ExampleWhy Class Exceptions?Built-in Exception ClassesSpecifying Exception TextSending Extra Data and Behavior in InstancesExample: Extra data with classes and stringsGeneral raise Statement FormsChapter Summary
Nesting Exception HandlersExample: Control-Flow NestingExample: Syntactic NestingException IdiomsExceptions Aren’t Always ErrorsFunctions Signal Conditions with raiseDebugging with Outer try StatementsRunning In-Process TestsMore on sys.exc_infoException Design TipsWhat Should Be WrappedCatching Too Much: Avoid Empty exceptsCatching Too Little: Use Class-Based CategoriesException GotchasString Exceptions Match by Identity, Not by ValueCatching the Wrong ThingCore Language SummaryThe Python ToolsetDevelopment Tools for Larger ProjectsChapter SummaryBRAIN BUILDERPart VII Exercises
Installing the Python InterpreterIs Python Already Present?Where to Fetch PythonInstallation StepsConfiguring PythonPython Environment VariablesHow to Set Configuration OptionsUnix/Linux shell variablesDOS variables (Windows)Other Windows optionsPath files
Part I, Getting StartedPart II, Types and OperationsPart III, Statements and SyntaxPart IV, FunctionsPart V, ModulesPart VI, Classes and OOPPart VII, Exceptions and Tools

Content preview from Learning Python, 3rd Edition

Chapter 18. Modules: The Big Picture

This chapter begins our in-depth look at the Python module, the highest-level program organization unit, which packages program code and data for reuse. In concrete terms, modules usually correspond to Python program files (or extensions coded in external languages such as C, Java, or C#). Each file is a module, and modules import other modules to use the names they define. Modules are processed with two statements, and one important built-in function:

import: Lets a client (importer) fetch a module as a whole
from: Allows clients to fetch particular names from a module
reload: Provides a way to reload a module’s code without stopping Python

Chapter 3 introduced module fundamentals, and we’ve been using them ever since. Part V begins by expanding on core module concepts, then moves on to explore more advanced module usage. This first chapter offers a general look at the role of modules in overall program structure. In the next and following chapters, we’ll dig into the coding details behind the theory.

Along the way, we’ll flesh out module details omitted so far: you’ll learn about reloads, the _ _name_ _ and _ _all_ _ attributes, package imports, and so on. Because modules and classes are really just glorified namespaces, we’ll formalize namespace concepts here as well.

Why Use Modules?

In short, modules provide an easy way to organize components into a system by serving as self-contained packages of variables known as namespaces. All the names defined ...