O'Reilly logo

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Python Cookbook, 2nd Edition

Book Description

Portable, powerful, and a breeze to use, Python is the popular open source object-oriented programming language used for both standalone programs and scripting applications. It is now being used by an increasing number of major organizations, including NASA and Google.Updated for Python 2.4, The Python Cookbook, 2nd Edition offers a wealth of useful code for all Python programmers, not just advanced practitioners. Like its predecessor, the new edition provides solutions to problems that Python programmers face everyday.It now includes over 200 recipes that range from simple tasks, such as working with dictionaries and list comprehensions, to complex tasks, such as monitoring a network and building a templating system. This revised version also includes new chapters on topics such as time, money, and metaprogramming.Here's a list of additional topics covered:

  • Manipulating text
  • Searching and sorting
  • Working with files and the filesystem
  • Object-oriented programming
  • Dealing with threads and processes
  • System administration
  • Interacting with databases
  • Creating user interfaces
  • Network and web programming
  • Processing XML
  • Distributed programming
  • Debugging and testing
Another advantage of The Python Cookbook, 2nd Edition is its trio of authors--three well-known Python programming experts, who are highly visible on email lists and in newsgroups, and speak often at Python conferences.With scores of practical examples and pertinent background information, The Python Cookbook, 2nd Edition is the one source you need if you're looking to build efficient, flexible, scalable, and well-integrated systems.

Table of Contents

  1. A Note Regarding Supplemental Files
  2. Preface
    1. The Design of the Book
    2. The Implementation of the Book
    3. Using the Code from This Book
    4. Audience
    5. Organization
    6. Further Reading
    7. Conventions Used in This Book
    8. How to Contact Us
    9. Safari® Enabled
    10. Acknowledgments
  3. 1. Text
    1. Introduction
    2. 1.1. Processing a String One Character at a Time
    3. 1.2. Converting Between Characters and Numeric Codes
    4. 1.3. Testing Whether an Object Is String-like
    5. 1.4. Aligning Strings
    6. 1.5. Trimming Space from the Ends of a String
    7. 1.6. Combining Strings
    8. 1.7. Reversing a String by Words or Characters
    9. 1.8. Checking Whether a String Contains a Set of Characters
    10. 1.9. Simplifying Usage of Strings’ translate Method
    11. 1.10. Filtering a String for a Set of Characters
    12. 1.11. Checking Whether a String Is Text or Binary
    13. 1.12. Controlling Case
    14. 1.13. Accessing Substrings
    15. 1.14. Changing the Indentation of a Multiline String
    16. 1.15. Expanding and Compressing Tabs
    17. 1.16. Interpolating Variables in a String
    18. 1.17. Interpolating Variables in a Stringin Python 2.4
    19. 1.18. Replacing Multiple Patterns in a Single Pass
    20. 1.19. Checking a String for Any of Multiple Endings
    21. 1.20. Handling International Text with Unicode
    22. 1.21. Converting Between Unicode and Plain Strings
    23. 1.22. Printing Unicode Charactersto Standard Output
    24. 1.23. Encoding Unicode Data for XML and HTML
    25. 1.24. Making Some Strings Case-Insensitive
    26. 1.25. Converting HTML Documents to Texton a Unix Terminal
  4. 2. Files
    1. Introduction
    2. 2.1. Reading from a File
    3. 2.2. Writing to a File
    4. 2.3. Searching and Replacing Text in a File
    5. 2.4. Reading a Specific Line from a File
    6. 2.5. Counting Lines in a File
    7. 2.6. Processing Every Word in a File
    8. 2.7. Using Random-Access Input/Output
    9. 2.8. Updating a Random-Access File
    10. 2.9. Reading Data from zip Files
    11. 2.10. Handling a zip File Inside a String
    12. 2.11. Archiving a Tree of Files into a Compressed tar File
    13. 2.12. Sending Binary Data to Standard Output Under Windows
    14. 2.13. Using a C++-like iostream Syntax
    15. 2.14. Rewinding an Input File to the Beginning
    16. 2.15. Adapting a File-like Object to a True File Object
    17. 2.16. Walking Directory Trees
    18. 2.17. Swapping One File Extension for Another Throughout a Directory Tree
    19. 2.18. Finding a File Given a Search Path
    20. 2.19. Finding Files Given a Search Path and a Pattern
    21. 2.20. Finding a File on the Python Search Path
    22. 2.21. Dynamically Changing the PythonSearch Path
    23. 2.22. Computing the Relative Path from One Directory to Another
    24. 2.23. Reading an Unbuffered Character in a Cross-Platform Way
    25. 2.24. Counting Pages of PDF Documents on Mac OS X
    26. 2.25. Changing File Attributes on Windows
    27. 2.26. Extracting Text from OpenOffice.org Documents
    28. 2.27. Extracting Text from Microsoft Word Documents
    29. 2.28. File Locking Using a Cross-Platform API
    30. 2.29. Versioning Filenames
    31. 2.30. Calculating CRC-64 Cyclic Redundancy Checks
  5. 3. Time and Money
    1. Introduction
    2. 3.1. Calculating Yesterday and Tomorrow
    3. 3.2. Finding Last Friday
    4. 3.3. Calculating Time Periods in a Date Range
    5. 3.4. Summing Durations of Songs
    6. 3.5. Calculating the Number of Weekdays Between Two Dates
    7. 3.6. Looking up Holidays Automatically
    8. 3.7. Fuzzy Parsing of Dates
    9. 3.8. Checking Whether Daylight Saving Time Is Currently in Effect
    10. 3.9. Converting Time Zones
    11. 3.10. Running a Command Repeatedly
    12. 3.11. Scheduling Commands
    13. 3.12. Doing Decimal Arithmetic
    14. 3.13. Formatting Decimals as Currency
    15. 3.14. Using Python as a Simple Adding Machine
    16. 3.15. Checking a Credit Card Checksum
    17. 3.16. Watching Foreign Exchange Rates
  6. 4. Python Shortcuts
    1. Introduction
    2. 4.1. Copying an Object
    3. 4.2. Constructing Lists with List Comprehensions
    4. 4.3. Returning an Element of a List If It Exists
    5. 4.4. Looping over Items and Their Indices in a Sequence
    6. 4.5. Creating Lists of Lists Without Sharing References
    7. 4.6. Flattening a Nested Sequence
    8. 4.7. Removing or Reordering Columnsin a List of Rows
    9. 4.8. Transposing Two-Dimensional Arrays
    10. 4.9. Getting a Value from a Dictionary
    11. 4.10. Adding an Entry to a Dictionary
    12. 4.11. Building a Dictionary Without Excessive Quoting
    13. 4.12. Building a Dict from a List of Alternating Keys and Values
    14. 4.13. Extracting a Subset of a Dictionary
    15. 4.14. Inverting a Dictionary
    16. 4.15. Associating Multiple Values with Each Key in a Dictionary
    17. 4.16. Using a Dictionary to Dispatch Methods or Functions
    18. 4.17. Finding Unions and Intersections of Dictionaries
    19. 4.18. Collecting a Bunch of Named Items
    20. 4.19. Assigning and Testing with One Statement
    21. 4.20. Using printf in Python
    22. 4.21. Randomly Picking Items with Given Probabilities
    23. 4.22. Handling Exceptions Within an Expression
    24. 4.23. Ensuring a Name Is Defined in a Given Module
  7. 5. Searching and Sorting
    1. Introduction
    2. 5.1. Sorting a Dictionary
    3. 5.2. Sorting a List of Strings Case-Insensitively
    4. 5.3. Sorting a List of Objects by an Attribute of the Objects
    5. 5.4. Sorting Keys or Indices Basedon the Corresponding Values
    6. 5.5. Sorting Strings with Embedded Numbers
    7. 5.6. Processing All of a List’s Items in Random Order
    8. 5.7. Keeping a Sequence Ordered as Items Are Added
    9. 5.8. Getting the First Few Smallest Items of a Sequence
    10. 5.9. Looking for Items in a Sorted Sequence
    11. 5.10. Selecting the nth Smallest Element of a Sequence
    12. 5.11. Showing off quicksort in Three Lines
    13. 5.12. Performing Frequent Membership Tests on a Sequence
    14. 5.13. Finding Subsequences
    15. 5.14. Enriching the Dictionary Type with Ratings Functionality
    16. 5.15. Sorting Names and Separating Them by Initials
  8. 6. Object-Oriented Programming
    1. Introduction
    2. 6.1. Converting Among Temperature Scales
    3. 6.2. Defining Constants
    4. 6.3. Restricting Attribute Setting
    5. 6.4. Chaining Dictionary Lookups
    6. 6.5. Delegating Automatically as an Alternative to Inheritance
    7. 6.6. Delegating Special Methods in Proxies
    8. 6.7. Implementing Tuples with Named Items
    9. 6.8. Avoiding Boilerplate Accessors for Properties
    10. 6.9. Making a Fast Copy of an Object
    11. 6.10. Keeping References to Bound Methods Without Inhibiting Garbage Collection
    12. 6.11. Implementing a Ring Buffer
    13. 6.12. Checking an Instance for Any State Changes
    14. 6.13. Checking Whether an Object Has Necessary Attributes
    15. 6.14. Implementing the State Design Pattern
    16. 6.15. Implementing the “Singleton” Design Pattern
    17. 6.16. Avoiding the “Singleton” Design Pattern with the Borg Idiom
    18. 6.17. Implementing the Null Object Design Pattern
    19. 6.18. Automatically Initializing Instance Variables from _ _init_ _ Arguments
    20. 6.19. Calling a Superclass _ _init_ _ Method If It Exists
    21. 6.20. Using Cooperative Supercalls Concisely and Safely
  9. 7. Persistence and Databases
    1. Introduction
    2. 7.1. Serializing Data Using the marshal Module
    3. 7.2. Serializing Data Using the pickle and cPickle Modules
    4. 7.3. Using Compression with Pickling
    5. 7.4. Using the cPickle Module on Classes and Instances
    6. 7.5. Holding Bound Methods in a Picklable Way
    7. 7.6. Pickling Code Objects
    8. 7.7. Mutating Objects with shelve
    9. 7.8. Using the Berkeley DB Database
    10. 7.9. Accessing a MySQL Database
    11. 7.10. Storing a BLOB in a MySQL Database
    12. 7.11. Storing a BLOB in a PostgreSQL Database
    13. 7.12. Storing a BLOB in a SQLite Database
    14. 7.13. Generating a Dictionary Mapping Field Names to Column Numbers
    15. 7.14. Using dtuple for Flexible Accessto Query Results
    16. 7.15. Pretty-Printing the Contents of Database Cursors
    17. 7.16. Using a Single Parameter-Passing Style Across Various DB API Modules
    18. 7.17. Using Microsoft Jet via ADO
    19. 7.18. Accessing a JDBC Database from a Jython Servlet
    20. 7.19. Using ODBC to Get Excel Data with Jython
  10. 8. Debugging and Testing
    1. Introduction
    2. 8.1. Disabling Execution of Some Conditionals and Loops
    3. 8.2. Measuring Memory Usage on Linux
    4. 8.3. Debugging the Garbage-Collection Process
    5. 8.4. Trapping and Recording Exceptions
    6. 8.5. Tracing Expressions and Comments in Debug Mode
    7. 8.6. Getting More Information from Tracebacks
    8. 8.7. Starting the Debugger Automatically After an Uncaught Exception
    9. 8.8. Running Unit Tests Most Simply
    10. 8.9. Running Unit Tests Automatically
    11. 8.10. Using doctest with unittest in Python 2.4
    12. 8.11. Checking Values Against Intervals in Unit Testing
  11. 9. Processes, Threads, and Synchronization
    1. Introduction
    2. 9.1. Synchronizing All Methods in an Object
    3. 9.2. Terminating a Thread
    4. 9.3. Using a Queue.Queue as a Priority Queue
    5. 9.4. Working with a Thread Pool
    6. 9.5. Executing a Function in Parallel on Multiple Argument Sets
    7. 9.6. Coordinating Threads by Simple Message Passing
    8. 9.7. Storing Per-Thread Information
    9. 9.8. Multitasking Cooperatively Without Threads
    10. 9.9. Determining Whether Another Instanceof a Script Is Already Running in Windows
    11. 9.10. Processing Windows Messages Using MsgWaitForMultipleObjects
    12. 9.11. Driving an External Process with popen
    13. 9.12. Capturing the Output and Error Streams from a Unix Shell Command
    14. 9.13. Forking a Daemon Process on Unix
  12. 10. System Administration
    1. Introduction
    2. 10.1. Generating Random Passwords
    3. 10.2. Generating Easily Remembered Somewhat-Random Passwords
    4. 10.3. Authenticating Users by Means of a POP Server
    5. 10.4. Calculating Apache Hits per IP Address
    6. 10.5. Calculating the Rate of Client Cache Hits on Apache
    7. 10.6. Spawning an Editor from a Script
    8. 10.7. Backing Up Files
    9. 10.8. Selectively Copying a Mailbox File
    10. 10.9. Building a Whitelist of Email Addresses From a Mailbox
    11. 10.10. Blocking Duplicate Mails
    12. 10.11. Checking Your Windows Sound System
    13. 10.12. Registering or Unregistering a DLL on Windows
    14. 10.13. Checking and Modifying the Set of Tasks Windows Automatically Runs at Login
    15. 10.14. Creating a Share on Windows
    16. 10.15. Connecting to an Already Running Instance of Internet Explorer
    17. 10.16. Reading Microsoft Outlook Contacts
    18. 10.17. Gathering Detailed System Informationon Mac OS X
  13. 11. User Interfaces
    1. Introduction
    2. 11.1. Showing a Progress Indicator on a Text Console
    3. 11.2. Avoiding lambda in Writing Callback Functions
    4. 11.3. Using Default Values and Bounds with tkSimpleDialog Functions
    5. 11.4. Adding Drag and Drop Reordering to a Tkinter Listbox
    6. 11.5. Entering Accented Characters in Tkinter Widgets
    7. 11.6. Embedding Inline GIFs Using Tkinter
    8. 11.7. Converting Among Image Formats
    9. 11.8. Implementing a Stopwatch in Tkinter
    10. 11.9. Combining GUIs and Asynchronous I/Owith Threads
    11. 11.10. Using IDLE’s Tree Widget in Tkinter
    12. 11.11. Supporting Multiple Values per Row in a Tkinter Listbox
    13. 11.12. Copying Geometry Methods and Options Between Tkinter Widgets
    14. 11.13. Implementing a Tabbed Notebook for Tkinter
    15. 11.14. Using a wxPython Notebook with Panels
    16. 11.15. Implementing an ImageJ Plug-in in Jython
    17. 11.16. Viewing an Image from a URL with Swing and Jython
    18. 11.17. Getting User Input on Mac OS
    19. 11.18. Building a Python Cocoa GUI Programmatically
    20. 11.19. Implementing Fade-in Windows with IronPython
  14. 12. Processing XML
    1. Introduction
    2. 12.1. Checking XML Well-Formedness
    3. 12.2. Counting Tags in a Document
    4. 12.3. Extracting Text from an XML Document
    5. 12.4. Autodetecting XML Encoding
    6. 12.5. Converting an XML Document into a Tree of Python Objects
    7. 12.6. Removing Whitespace-only Text Nodes from an XML DOM Node’s Subtree
    8. 12.7. Parsing Microsoft Excel’s XML
    9. 12.8. Validating XML Documents
    10. 12.9. Filtering Elements and Attributes Belonging to a Given Namespace
    11. 12.10. Merging Continuous Text Events with a SAX Filter
    12. 12.11. Using MSHTML to Parse XML or HTML
  15. 13. Network Programming
    1. Introduction
    2. 13.1. Passing Messages with Socket Datagrams
    3. 13.2. Grabbing a Document from the Web
    4. 13.3. Filtering a List of FTP Sites
    5. 13.4. Getting Time from a Server via the SNTP Protocol
    6. 13.5. Sending HTML Mail
    7. 13.6. Bundling Files in a MIME Message
    8. 13.7. Unpacking a Multipart MIME Message
    9. 13.8. Removing Attachments from an Email Message
    10. 13.9. Fixing Messages Parsed by Python 2.4 email.FeedParser
    11. 13.10. Inspecting a POP3 Mailbox Interactively
    12. 13.11. Detecting Inactive Computers
    13. 13.12. Monitoring a Network with HTTP
    14. 13.13. Forwarding and Redirecting Network Ports
    15. 13.14. Tunneling SSL Through a Proxy
    16. 13.15. Implementing the Dynamic IP Protocol
    17. 13.16. Connecting to IRC and Logging Messages to Disk
    18. 13.17. Accessing LDAP Servers
  16. 14. Web Programming
    1. Introduction
    2. 14.1. Testing Whether CGI Is Working
    3. 14.2. Handling URLs Within a CGI Script
    4. 14.3. Uploading Files with CGI
    5. 14.4. Checking for a Web Page’s Existence
    6. 14.5. Checking Content Type via HTTP
    7. 14.6. Resuming the HTTP Download of a File
    8. 14.7. Handling Cookies While Fetching Web Pages
    9. 14.8. Authenticating with a Proxy for HTTPS Navigation
    10. 14.9. Running a Servlet with Jython
    11. 14.10. Finding an Internet Explorer Cookie
    12. 14.11. Generating OPML Files
    13. 14.12. Aggregating RSS Feeds
    14. 14.13. Turning Data into Web Pages Through Templates
    15. 14.14. Rendering Arbitrary Objects with Nevow
  17. 15. Distributed Programming
    1. Introduction
    2. 15.1. Making an XML-RPC Method Call
    3. 15.2. Serving XML-RPC Requests
    4. 15.3. Using XML-RPC with Medusa
    5. 15.4. Enabling an XML-RPC Server to Be Terminated Remotely
    6. 15.5. Implementing SimpleXMLRPCServer Niceties
    7. 15.6. Giving an XML-RPC Server a wxPython GUI
    8. 15.7. Using Twisted Perspective Broker
    9. 15.8. Implementing a CORBA Server and Client
    10. 15.9. Performing Remote Logins Using telnetlib
    11. 15.10. Performing Remote Logins with SSH
    12. 15.11. Authenticating an SSL Client over HTTPS
  18. 16. Programs About Programs
    1. Introduction
    2. 16.1. Verifying Whether a String Represents a Valid Number
    3. 16.2. Importing a Dynamically Generated Module
    4. 16.3. Importing from a Module Whose Name Is Determined at Runtime
    5. 16.4. Associating Parameters with a Function (Currying)
    6. 16.5. Composing Functions
    7. 16.6. Colorizing Python Source Using the Built-in Tokenizer
    8. 16.7. Merging and Splitting Tokens
    9. 16.8. Checking Whether a String Has Balanced Parentheses
    10. 16.9. Simulating Enumerations in Python
    11. 16.10. Referring to a List Comprehension While Building It
    12. 16.11. Automating the py2exe Compilation of Scripts into Windows Executables
    13. 16.12. Binding Main Script and Modules into One Executable on Unix
  19. 17. Extending and Embedding
    1. Introduction
    2. 17.1. Implementing a Simple Extension Type
    3. 17.2. Implementing a Simple Extension Type with Pyrex
    4. 17.3. Exposing a C++ Library to Python
    5. 17.4. Calling Functions from a Windows DLL
    6. 17.5. Using SWIG-Generated Modules in a Multithreaded Environment
    7. 17.6. Translating a Python Sequence into a C Array with the PySequence_Fast Protocol
    8. 17.7. Accessing a Python Sequence Item-by-Item with the Iterator Protocol
    9. 17.8. Returning None from a Python-Callable C Function
    10. 17.9. Debugging Dynamically Loaded C Extensions with gdb
    11. 17.10. Debugging Memory Problems
  20. 18. Algorithms
    1. Introduction
    2. 18.1. Removing Duplicates from a Sequence
    3. 18.2. Removing Duplicates from a Sequence While Maintaining Sequence Order
    4. 18.3. Generating Random Samples with Replacement
    5. 18.4. Generating Random Samples Without Replacement
    6. 18.5. Memoizing (Caching) the Return Values of Functions
    7. 18.6. Implementing a FIFO Container
    8. 18.7. Caching Objects with a FIFO Pruning Strategy
    9. 18.8. Implementing a Bag (Multiset) Collection Type
    10. 18.9. Simulating the Ternary Operator in Python
    11. 18.10. Computing Prime Numbers
    12. 18.11. Formatting Integers as Binary Strings
    13. 18.12. Formatting Integers as Strings in Arbitrary Bases
    14. 18.13. Converting Numbers to Rationals via Farey Fractions
    15. 18.14. Doing Arithmetic with Error Propagation
    16. 18.15. Summing Numbers with Maximal Accuracy
    17. 18.16. Simulating Floating Point
    18. 18.17. Computing the Convex Hulls and Diameters of 2D Point Sets
  21. 19. Iterators and Generators
    1. Introduction
    2. 19.1. Writing a range-like Function with Float Increments
    3. 19.2. Building a List from Any Iterable
    4. 19.3. Generating the Fibonacci Sequence
    5. 19.4. Unpacking a Few Items in a Multiple Assignment
    6. 19.5. Automatically Unpacking the Needed Number of Items
    7. 19.6. Dividing an Iterable into n Slices of Stride n
    8. 19.7. Looping on a Sequence by Overlapping Windows
    9. 19.8. Looping Through Multiple Iterables in Parallel
    10. 19.9. Looping Through the Cross-Product of Multiple Iterables
    11. 19.10. Reading a Text File by Paragraphs
    12. 19.11. Reading Lines with Continuation Characters
    13. 19.12. Iterating on a Stream of Data Blocks as a Stream of Lines
    14. 19.13. Fetching Large Record Sets from a Database with a Generator
    15. 19.14. Merging Sorted Sequences
    16. 19.15. Generating Permutations, Combinations, and Selections
    17. 19.16. Generating the Partitions of an Integer
    18. 19.17. Duplicating an Iterator
    19. 19.18. Looking Ahead into an Iterator
    20. 19.19. Simplifying Queue-Consumer Threads
    21. 19.20. Running an Iterator in Another Thread
    22. 19.21. Computing a Summary Report with itertools.groupby
  22. 20. Descriptors, Decorators,and Metaclasses
    1. Introduction
    2. 20.1. Getting Fresh Default Values at Each Function Call
    3. 20.2. Coding Properties as Nested Functions
    4. 20.3. Aliasing Attribute Values
    5. 20.4. Caching Attribute Values
    6. 20.5. Using One Method as Accessorfor Multiple Attributes
    7. 20.6. Adding Functionality to a Class by Wrapping a Method
    8. 20.7. Adding Functionality to a Class by Enriching All Methods
    9. 20.8. Adding a Method to a Class Instance at Runtime
    10. 20.9. Checking Whether Interfaces Are Implemented
    11. 20.10. Using _ _new_ _ and _ _init_ _ Appropriately in Custom Metaclasses
    12. 20.11. Allowing Chaining of Mutating List Methods
    13. 20.12. Using Cooperative Super calls with Terser Syntax
    14. 20.13. Initializing Instance Attributes Without Using _ _init_ _
    15. 20.14. Automatic Initialization of Instance Attributes
    16. 20.15. Upgrading Class Instances Automatically on reload
    17. 20.16. Binding Constants at Compile Time
    18. 20.17. Solving Metaclass Conflicts
  23. Index
  24. About the Authors
  25. Colophon
  26. Copyright