book

Introducing Python

by Bill Lubanovic

November 2014

Beginner

481 pages

10h 11m

English

O'Reilly Media, Inc.

Read now

Unlock full access

AudienceOutlinePython VersionsConventions Used in This BookUsing Code ExamplesSafari® Books OnlineHow to Contact UsAcknowledgments
Python in the Real WorldPython versus Language XSo, Why Python?When Not to Use PythonPython 2 versus Python 3Installing PythonRunning PythonUsing the Interactive InterpreterUse Python FilesWhat’s Next?Your Moment of ZenThings to Do
Variables, Names, and ObjectsNumbersIntegersPrecedenceBasesType ConversionsHow Big Is an int?FloatsMath FunctionsStringsCreate with QuotesConvert Data Types by Using str()Escape with \Combine with +Duplicate with *Extract a Character with []Slice with [ start : end : step ]Get Length with len()Split with split()Combine with join()Playing with StringsCase and AlignmentSubstitute with replace()More String ThingsThings to Do
Lists and TuplesListsCreate with [] or list()Convert Other Data Types to Lists with list()Get an Item by Using [ offset ]Lists of ListsChange an Item by [ offset ]Get a Slice to Extract Items by Offset RangeAdd an Item to the End with append()Combine Lists by Using extend() or +=Add an Item by Offset with insert()Delete an Item by Offset with delDelete an Item by Value with remove()Get an Item by Offset and Delete It by Using pop()Find an Item’s Offset by Value with index()Test for a Value with inCount Occurrences of a Value by Using count()Convert to a String with join()Reorder Items with sort()Get Length by Using len()Assign with =, Copy with copy()TuplesCreate a Tuple by Using ()Tuples versus ListsDictionariesCreate with {}Convert by Using dict()Add or Change an Item by [ key ]Combine Dictionaries with update()Delete an Item by Key with delDelete All Items by Using clear()Test for a Key by Using inGet an Item by [ key ]Get All Keys by Using keys()Get All Values by Using values()Get All Key-Value Pairs by Using items()Assign with =, Copy with copy()SetsCreate with set()Convert from Other Data Types with set()Test for Value by Using inCombinations and OperatorsCompare Data StructuresMake Bigger Data StructuresThings to Do
Comment with #Continue Lines with \Compare with if, elif, and elseWhat Is True?Do Multiple Comparisons with inRepeat with whileCancel with breakSkip Ahead with continueCheck break Use with elseIterate with forCancel with breakSkip with continueCheck break Use with elseIterate Multiple Sequences with zip()Generate Number Sequences with range()Other IteratorsComprehensionsList ComprehensionsDictionary ComprehensionsSet ComprehensionsGenerator ComprehensionsFunctionsPositional ArgumentsKeyword ArgumentsSpecify Default Parameter ValuesGather Positional Arguments with *Gather Keyword Arguments with **DocstringsFunctions Are First-Class CitizensInner FunctionsClosuresAnonymous Functions: the lambda() FunctionGeneratorsDecoratorsNamespaces and ScopeUses of _ and __ in NamesHandle Errors with try and exceptMake Your Own ExceptionsThings to Do
Standalone ProgramsCommand-Line ArgumentsModules and the import StatementImport a ModuleImport a Module with Another NameImport Only What You Want from a ModuleModule Search PathPackagesThe Python Standard LibraryHandle Missing Keys with setdefault() and defaultdict()Count Items with Counter()Order by Key with OrderedDict()Stack + Queue == dequeIterate over Code Structures with itertoolsPrint Nicely with pprint()More Batteries: Get Other Python CodeThings to Do
What Are Objects?Define a Class with classInheritanceOverride a MethodAdd a MethodGet Help from Your Parent with superIn self DefenseGet and Set Attribute Values with PropertiesName Mangling for PrivacyMethod TypesDuck TypingSpecial MethodsAggregation and CompositionWhen to Use Classes and Objects versus ModulesNamed TuplesThings to Do
Text StringsUnicodeFormatMatch with Regular ExpressionsBinary Databytes and bytearrayConvert Binary Data with structOther Binary Data ToolsConvert Bytes/Strings with binascii()Bit OperatorsThings to Do
File Input/OutputWrite a Text File with write()Read a Text File with read(), readline(), or readlines()Write a Binary File with write()Read a Binary File with read()Close Files Automatically by Using withChange Position with seek()Structured Text FilesCSVXMLHTMLJSONYAMLA Security NoteConfiguration FilesOther Interchange FormatsSerialize by Using pickleStructured Binary FilesSpreadsheetsHDF5Relational DatabasesSQLDB-APISQLiteMySQLPostgreSQLSQLAlchemyNoSQL Data StoresThe dbm FamilyMemcachedRedisOther NoSQLFull-Text DatabasesThings to Do
Web ClientsTest with telnetPython’s Standard Web LibrariesBeyond the Standard Library: RequestsWeb ServersThe Simplest Python Web ServerWeb Server Gateway InterfaceFrameworksBottleFlaskNon-Python Web ServersOther FrameworksWeb Services and AutomationThe webbrowser ModuleWeb APIs and Representational State TransferJSONCrawl and ScrapeScrape HTML with BeautifulSoupThings to Do

FilesCreate with open()Check Existence with exists()Check Type with isfile()Copy with copy()Change Name with rename()Link with link() or symlink()Change Permissions with chmod()Change Ownership with chown()Get a Pathname with abspath()Get a symlink Pathname with realpath()Delete a File with remove()DirectoriesCreate with mkdir()Delete with rmdir()List Contents with listdir()Change Current Directory with chdir()List Matching Files with glob()Programs and ProcessesCreate a Process with subprocessCreate a Process with multiprocessingKill a Process with terminate()Calendars and ClocksThe datetime ModuleUsing the time ModuleRead and Write Dates and TimesAlternative ModulesThings to Do
ConcurrencyQueuesProcessesThreadsGreen Threads and geventtwistedasyncioRedisBeyond QueuesNetworksPatternsThe Publish-Subscribe ModelTCP/IPSocketsZeroMQScapyInternet ServicesWeb Services and APIsRemote ProcessingBig Fat Data and MapReduceWorking in the CloudsThings to Do
About ProgrammingFind Python CodeInstall PackagesUse pipUse a Package ManagerInstall from SourceIntegrated Development EnvironmentsIDLEPyCharmIPythonName and DocumentTesting Your CodeCheck with pylint, pyflakes, and pep8Test with unittestTest with doctestTest with noseOther Test FrameworksContinuous IntegrationDebugging Python CodeDebug with pdbLogging Error MessagesOptimize Your CodeMeasure TimingAlgorithms and Data StructuresCython, NumPy, and C ExtensionsPyPySource ControlMercurialGitClone This BookHow You Can Learn MoreBooksWebsitesGroupsConferencesComing Attractions
2-D GraphicsStandard LibraryPIL and PillowImageMagickGraphical User Interfaces (GUIs)3-D Graphics and AnimationPlots, Graphs, and VisualizationmatplotlibbokehGamesAudio and Music
The Microsoft Office SuiteCarrying Out Business TasksProcessing Business DataExtracting, Transforming, and LoadingAdditional Sources of InformationPython in FinanceBusiness Data SecurityMapsFormatsDraw a MapApplications and Data
Math and Statistics in the Standard LibraryMath FunctionsWorking with Complex NumbersCalculate Accurate Floating Point with decimalPerform Rational Arithmetic with fractionsUse Packed Sequences with arrayHandling Simple Stats by Using statisticsMatrix MultiplicationScientific PythonNumPyMake an Array with array()Make an Array with arange()Make an Array with zeros(), ones(), or random()Change an Array’s Shape with reshape()Get an Element with []Array MathLinear AlgebraThe SciPy LibraryThe SciKit LibraryThe IPython LibraryA Better InterpreterIPython NotebookPandasPython and Scientific Areas
Install Standard PythonMac OS XWindowsLinux or UnixInstall AnacondaInstall and Use pip and virtualenvInstall and Use conda
Chapter 1, A Taste of PyChapter 2, Py Ingredients: Numbers, Strings, and VariablesChapter 3, Py Filling: Lists, Tuples, Dictionaries, and SetsChapter 4, Py Crust: Code StructuresChapter 5, Py Boxes: Modules, Packages, and ProgramsChapter 6, Oh Oh: Objects and ClassesChapter 7, Mangle Data Like a ProChapter 8, Data Has to Go SomewhereChapter 9, The Web, UntangledChapter 10, SystemsChapter 11, Concurrency and Networks
Operator PrecedenceString MethodsChange CaseSearchModifyFormatString TypeString Module AttributesFin

Content preview from Introducing Python

Chapter 8. Data Has to Go Somewhere

It is a capital mistake to theorize before one has data.

Arthur Conan Doyle

An active program accesses data that is stored in Random Access Memory, or RAM. RAM is very fast, but it is expensive and requires a constant supply of power; if the power goes out, all the data in memory is lost. Disk drives are slower than RAM but have more capacity, cost less, and retain data even after someone trips over the power cord. Thus, a huge amount of effort in computer systems has been devoted to making the best tradeoffs between storing data on disk and RAM. As programmers, we need persistence: storing and retrieving data using nonvolatile media such as disks.

This chapter is all about the different flavors of data storage, each optimized for different purposes: flat files, structured files, and databases. File operations other than input and output are covered in “Files”.

Note

This is also the first chapter to show examples of nonstandard Python modules; that is, Python code apart from the standard library. You’ll install them by using the pip command, which is painless. There are more details on its usage in Appendix D.

File Input/Output

The simplest kind of persistence is a plain old file, sometimes called a flat file. This is just a sequence of bytes stored under a filename. You read from a file into memory and write from memory to a file. Python makes these jobs easy. Its file operations were modeled on the familiar and popular Unix equivalents. ...