Buy this Book
Print Book $29.95
Add to Cart
Read it Now!
Print Book £20.95
Add to UK Cart
Reprint Licensing
Envelope Tell a friend

Python Standard Library
Python Standard Library

By Fredrik Lundh
Price: $29.95 USD
£20.95 GBP

Cover | Table of Contents | Colophon

Table of Contents

Chapter 1: Core Modules
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
"Since the functions in the C runtime library are not part of the Win32 API, we believe the number of applications that will be affected by this bug to be very limited."
—Microsoft, January 1999
Python's standard library covers a wide range of modules. It includes everything from modules that are as much a part of the Python language as the types and statements defined by the language specification, to obscure modules that are probably useful only to a small number of programs.
This chapter describes a number of fundamental standard library modules. Any larger Python program is likely to use most of these modules, either directly or indirectly.
The following two modules are even more basic than all other modules combined: the _ _builtin_ _ module, which defines built-in functions (like len, int, and range), and the exceptions module, which defines all built-in exceptions.
Python imports both modules when it starts up, and makes their content available for all programs.
There are a number of modules modeled after the POSIX standard API and the standard C library that provide platform-independent interfaces to the underlying operating system.
The modules in this group include os, which provides file and process operations, os.path, which offers a platform-independent way to pull apart and put together filenames, and time, which provides functions to work with dates and times.
To some extent, networking and thread support modules could also belong in this group, but they are not supported by all Python implementations.
Several built-in types have support modules in the standard library. The string module implements commonly used string operations, the math module provides math operations and constants, and the cmath module does the same for complex numbers.
The re module provides regular expressions support for Python. Regular expressions are string patterns written in a special syntax, which can be used to match strings and extract substrings.
sys gives you access to various interpreter variables, such as the module search path, and the interpreter version.
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
Introduction
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
Python's standard library covers a wide range of modules. It includes everything from modules that are as much a part of the Python language as the types and statements defined by the language specification, to obscure modules that are probably useful only to a small number of programs.
This chapter describes a number of fundamental standard library modules. Any larger Python program is likely to use most of these modules, either directly or indirectly.
The following two modules are even more basic than all other modules combined: the _ _builtin_ _ module, which defines built-in functions (like len, int, and range), and the exceptions module, which defines all built-in exceptions.
Python imports both modules when it starts up, and makes their content available for all programs.
There are a number of modules modeled after the POSIX standard API and the standard C library that provide platform-independent interfaces to the underlying operating system.
The modules in this group include os, which provides file and process operations, os.path, which offers a platform-independent way to pull apart and put together filenames, and time, which provides functions to work with dates and times.
To some extent, networking and thread support modules could also belong in this group, but they are not supported by all Python implementations.
Several built-in types have support modules in the standard library. The string module implements commonly used string operations, the math module provides math operations and constants, and the cmath module does the same for complex numbers.
The re module provides regular expressions support for Python. Regular expressions are string patterns written in a special syntax, which can be used to match strings and extract substrings.
sys gives you access to various interpreter variables, such as the module search path, and the interpreter version. operator provides functional equivalents to many built-in operators. copy allows you to copy objects. And finally,
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The _ _builtin_ _ Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
This module contains built-in functions that are automatically available in all Python modules. You usually don't have to import this module; Python does that for you when necessary.
Python allows you to build function argument lists on the fly. Just put all the arguments in a tuple, and call the built-in apply function, as illustrated in Example 1-1.
Example 1-1. Using the apply Function
File: builtin-apply-example-1.py

def function(a, b):
    print a, b

apply(function, ("whither", "canada?"))
apply(function, (1, 2 + 3))

whither canada?
1 5
To pass keyword arguments to a function, you can use a dictionary as the third argument to apply, as shown in Example 1-2.
Example 1-2. Using the apply Function to Pass Keyword Arguments
File: builtin-apply-example-2.py

def function(a, b):
    print a, b

apply(function, ("crunchy", "frog"))
apply(function, ("crunchy",), {"b": "frog"})
apply(function, (), {"a": "crunchy", "b": "frog"})

crunchy frog
crunchy frog
crunchy frog
One common use for apply is to pass constructor arguments from a subclass on to the base class, especially if the constructor takes a lot of arguments. See Example 1-3.
Example 1-3. Using the apply Function to Call Base Class Constructors
File: builtin-apply-example-3.py

class Rectangle:
    def _ _init_ _(self, color="white", width=10, height=10):
        print "create a", color, self, "sized", width, "x", height

class RoundedRectangle(Rectangle):
    def _ _init_ _(self, **kw):
        apply(Rectangle._ _init_ _, (self,), kw)

rect = Rectangle(color="green", height=100, width=100)
rect = RoundedRectangle(color="blue", height=20)

create a green <Rectangle instance at 8c8260> sized 100 x 100
create a blue <RoundedRectangle instance at 8c84c0> sized 10 x 20
Python 2.0 provides an alternate syntax. Instead of apply, you can use an ordinary function call, and use * to mark the tuple, and ** to mark the dictionary.
The following two statements are equivalent:
result = function(*args, **kwargs)
result = apply(function, args, kwargs)
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The exceptions Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The exceptions module provides the standard exception hierarchy. It's automatically imported when Python starts, and the exceptions are added to the _ _builtin_ _ module. In other words, you usually don't need to import this module.
This is a Python module in 1.5.2, and a built-in module in 2.0 and later.
The following standard exceptions are defined by this module:
  • Exception is used as a base class for all exceptions. It's strongly recommended (but not yet required) that user exceptions are derived from this class too.
  • SystemExit(Exception) is raised by the sys.exit function. If it propagates to the top level without being caught by a try-except clause, the interpreter is terminated without a traceback message.
  • StandardError(Exception) is used as a base class for all standard exceptions (except SystemExit, that is).
  • KeyboardInterrupt(StandardError) is raised when the user presses Control-C (or any other interrupt key). Note that this may cause strange errors if you use "catch all" try-except statements.
  • ImportError(StandardError) is raised when Python fails to import a module.
  • EnvironmentError is used as a base class for exceptions that can be caused by the interpreter's environment (that is, they're usually not caused by bugs in the program).
  • IOError(EnvironmentError) is used to flag I/O-related errors.
  • OSError(EnvironmentError) is used to flag errors by the os module.
  • WindowsError(OSError) is used to flag Windows-specific errors from the os module.
  • NameError(StandardError) is raised when Python fails to find a global or local name.
  • UnboundLocalError(NameError) is raised if your program attempts to access a local variable before it has been assigned a value. This exception is only used in 2.0 and later; earlier versions raise a plain NameError exception instead.
  • AttributeError(StandardError) is raised when Python fails to find (or assign to) an instance attribute, a method, a module function, or any other qualified name.
  • SyntaxError(StandardError) is raised when the compiler stumbles upon a syntax error.
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The os Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The os module provides a unified interface to many operating system functions.
Most of the functions in this module are implemented by platform-specific modules, such as posix or nt. The os module automatically loads the right implementation module when it is first imported.
The built-in open function lets you create, open, and modify files, as shown in Example 1-27. This module adds those extra functions you need to rename and remove files.
Example 1-27. Using the os Module to Rename and Remove Files
File: os-example-3.py

import os
import string

def replace(file, search_for, replace_with):
    # replace strings in a text file

    back = os.path.splitext(file)[0] + ".bak"
    temp = os.path.splitext(file)[0] + ".tmp"

    try:
        # remove old temp file, if any
        os.remove(temp)
    except os.error:
        pass

    fi = open(file)
    fo = open(temp, "w")

    for s in fi.readlines():
        fo.write(string.replace(s, search_for, replace_with))

    fi.close()
    fo.close()

    try:
        # remove old backup file, if any
        os.remove(back)
    except os.error:
        pass

    # rename original to backup...
    os.rename(file, back)

    # ...and temporary to original
    os.rename(temp, file)

#
# try it out!

file = "samples/sample.txt"

replace(file, "hello", "tjena")
replace(file, "tjena", "hello")
The os module also contains many functions that work on entire directories.
The listdir function returns a list of all filenames in a given directory, as shown in Example 1-28. The current and parent directory markers used on Unix and Windows (. and ..) are not included in this list.
Example 1-28. Using the os Module to List the Files in a Directory
File: os-example-5.py

import os

for file in os.listdir("samples"):
    print file

sample.au
sample.jpg
sample.wav
...
The getcwd and chdir functions are used to get and set the current directory, as shown in Example 1-29.
Example 1-29. Using the os Module to Change the Working Directory
File: os-example-4.py

import os

# where are we?
cwd = os.getcwd()
print "1", cwd

# go down
os.chdir("samples")
print "2", os.getcwd()

# go back up
os.chdir(os.pardir)
print "3", os.getcwd()
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The os.path Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The os.path module contains functions that deal with long filenames (pathnames) in various ways. To use this module, import the os module, and access this module as os.path.
The os.path module contains a number of functions that deal with long filenames in a platform independent way. In other words, you won't have to deal with forward and backward slashes, colons, and whatnot. Let's look at Example 1-42.
Example 1-42. Using the os.path Module to Handle Filename
File: os-path-example-1.py

import os

filename = "my/little/pony"

print "using", os.name, "..."
print "split", "=>", os.path.split(filename)
print "splitext", "=>", os.path.splitext(filename)
print "dirname", "=>", os.path.dirname(filename)
print "basename", "=>", os.path.basename(filename)
print "join", "=>", os.path.join(os.path.dirname(filename),
                                 os.path.basename(filename))

using nt ...
split => ('my/little', 'pony')
splitext => ('my/little/pony', '')
dirname => my/little
basename => pony
join => my/little\pony
Note that split only splits off a single item.
The os.path module also contains a number of functions that allow you to quickly figure out what a filename represents, as shown in Example 1-43.
Example 1-43. Using the os.path Module to Check What a Filename Represents
File: os-path-example-2.py

import os

FILES = (
    os.curdir,
    "/",
    "file",
    "/file",
    "samples",
    "samples/sample.jpg",
    "directory/file",
    "../directory/file",
    "/directory/file"
    )

for file in FILES:
    print file, "=>",
    if os.path.exists(file):
        print "EXISTS",
    if os.path.isabs(file):
        print "ISABS",
    if os.path.isdir(file):
        print "ISDIR",
    if os.path.isfile(file):
        print "ISFILE",
    if os.path.islink(file):
        print "ISLINK",
    if os.path.ismount(file):
        print "ISMOUNT",
    print

. => EXISTS ISDIR
/ => EXISTS ISABS ISDIR ISMOUNT
file =>
/file => ISABS
samples => EXISTS ISDIR
samples/sample.jpg => EXISTS ISFILE
directory/file =>
../directory/file =>
/directory/file => ISABS
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The stat Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The stat module, shown in Example 1-50, contains a number of constants and test functions that can be used with the os.stat function.
Example 1-50. Using the stat Module
File: stat-example-1.py

import stat
import os, time

st = os.stat("samples/sample.txt")

print "mode", "=>", oct(stat.S_IMODE(st[stat.ST_MODE]))

print "type", "=>",
if stat.S_ISDIR(st[stat.ST_MODE]):
    print "DIRECTORY",
if stat.S_ISREG(st[stat.ST_MODE]):
    print "REGULAR",
if stat.S_ISLNK(st[stat.ST_MODE]):
    print "LINK",
print

print "size", "=>", st[stat.ST_SIZE]

print "last accessed", "=>", time.ctime(st[stat.ST_ATIME])
print "last modified", "=>", time.ctime(st[stat.ST_MTIME])
print "inode changed", "=>", time.ctime(st[stat.ST_CTIME])

mode => 0664
type => REGULAR
size => 305
last accessed => Sun Oct 10 22:12:30 1999
last modified => Sun Oct 10 18:39:37 1999
inode changed => Sun Oct 10 15:26:38 1999
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The string Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The string module contains a number of functions to process standard Python strings, as shown in Example 1-51.
Example 1-51. Using the string Module
File: string-example-1.py

import string

text = "Monty Python's Flying Circus"

print "upper", "=>", string.upper(text)
print "lower", "=>", string.lower(text)
print "split", "=>", string.split(text)
print "join", "=>", string.join(string.split(text), "+")
print "replace", "=>", string.replace(text, "Python", "Java")
print "find", "=>", string.find(text, "Python"), string.find(text, "Java")
print "count", "=>", string.count(text, "n")

upper => MONTY PYTHON'S FLYING CIRCUS
lower => monty python's flying circus
split => ['Monty', "Python's", 'Flying', 'Circus']
join => Monty+Python's+Flying+Circus
replace => Monty Java's Flying Circus
find => 6 -1
count => 3
In Python 1.5.2 and earlier, the string module uses functions from the strop implementation module where possible.
In Python 1.6 and later, most string operations are made available as string methods as well, as shown in Example 1-52. Many of the functions in the string module are simply wrapper functions that call the corresponding string method.
Example 1-52. Using string Methods Instead of string Module Functions
File: string-example-2.py

text = "Monty Python's Flying Circus"

print "upper", "=>", text.upper()
print "lower", "=>", text.lower()
print "split", "=>", text.split()
print "join", "=>", "+".join(text.split())
print "replace", "=>", text.replace("Python", "Perl")
print "find", "=>", text.find("Python"), text.find("Perl")
print "count", "=>", text.count("n")

upper => MONTY PYTHON'S FLYING CIRCUS
lower => monty python's flying circus
split => ['Monty', "Python's", 'Flying', 'Circus']
join => Monty+Python's+Flying+Circus
replace => Monty Perl's Flying Circus
find => 6 -1
count => 3
In addition to the string-manipulation capabilities offered by string, the module also contains a number of functions that convert strings to other types (as Example 1-53 demonstrates).
Example 1-53. Using the string Module to Convert Strings to Numbers
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The re Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
"Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems."
—Jamie Zawinski, on comp.lang.emacs
The re module provides a set of powerful regular expression facilities, which allows you to quickly check whether a given string matches a given pattern (using the match function), or contains such a pattern (using the search function). A regular expression is a string pattern written in a compact (and quite cryptic) syntax.
The match function attempts to match a pattern against the beginning of the given string, as shown in Example 1-54. If the pattern matches anything at all (including an empty string, if the pattern allows that!), match returns a match object. The group method can be used to find out what matched.
Example 1-54. Using the re Module to Match Strings
File: re-example-1.py

import re

text = "The Attila the Hun Show"

# a single character
m = re.match(".", text)
if m: print repr("."), "=>", repr(m.group(0))

# any string of characters
m = re.match(".*", text)
if m: print repr(".*"), "=>", repr(m.group(0))

# a string of letters (at least one)
m = re.match("\w+", text)
if m: print repr("\w+"), "=>", repr(m.group(0))

# a string of digits
m = re.match("\d+", text)
if m: print repr("\d+"), "=>", repr(m.group(0))

 '.' => 'T'
'.*' => 'The Attila the Hun Show'
'\\w+' => 'The'
You can use parentheses to mark regions in the pattern. If the pattern matched, the group method can be used to extract the contents of these regions, as shown in Example 1-55. group(1) returns the contents of the first group, group(2) returns the contents of the second, and so on. If you pass several group numbers to the group function, it returns a tuple.
Example 1-55. Using the re Module to Extract Matching Substrings
File: re-example-2.py

import re

text ="10/15/99"

m = re.match("(\d{2})/(\d{2})/(\d{2,4})", text)
if m:
    print m.group(1, 2, 3)

('10', '15', '99')
The search function searches for the pattern inside the string, as shown in Example 1-56. It basically tries the pattern at every possible character position, starting from the left, and returns a match object as soon it has found a match. If the pattern doesn't match anywhere, it returns
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The math Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The math module implements a number of mathematical operations for floating-point numbers. The functions are generally thin wrappers around the platform C library functions of the same name, so results may vary slightly across platforms in normal cases, or vary a lot in exceptional cases. Example 1-60 demonstrates the use of the math module.
Example 1-60. Using the math Module
File: math-example-1.py

import math

print "e", "=>", math.e
print "pi", "=>", math.pi
print "hypot", "=>", math.hypot(3.0, 4.0)

# and many others...

e => 2.71828182846
pi => 3.14159265359
hypot => 5.0
See the Python Library Reference for a full list of functions.
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The cmath Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The cmath module shown in Example 1-61 contains a number of mathematical operations for complex numbers.
Example 1-61. Using the cmath Module
File: cmath-example-1.py

import cmath

print "pi", "=>", cmath.pi
print "sqrt(-1)", "=>", cmath.sqrt(-1)

pi => 3.14159265359
sqrt(-1) => 1j
See the Python Library Reference for a full list of functions.
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The operator Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The operator module provides a "functional" interface to the standard operators in Python. The functions in this module can be used instead of some lambda constructs, when processing data with functions like map and filter. They are also quite popular among people who like to write obscure code, for obvious reasons. The operator module is demonstrated in Example 1-62.
Example 1-62. Using the operator Module
File: operator-example-1.py

import operator

sequence = 1, 2, 4

print "add", "=>", reduce(operator.add, sequence)
print "sub", "=>", reduce(operator.sub, sequence)
print "mul", "=>", reduce(operator.mul, sequence)
print "concat", "=>", operator.concat("spam", "egg")
print "repeat", "=>", operator.repeat("spam", 5)
print "getitem", "=>", operator.getitem(sequence, 2)
print "indexOf", "=>", operator.indexOf(sequence, 2)
print "sequenceIncludes", "=>", operator.sequenceIncludes(sequence, 3)

add => 7
sub => -5
mul => 8
concat => spamegg
repeat => spamspamspamspamspam

getitem => 4
indexOf => 1
sequenceIncludes => 0
Example 1-63 shows some operator functions that can be used to check object types.
Example 1-63. Using the operator Module for Type Checking
File: operator-example-2.py

import operator
import UserList

def dump(data):
    print type(data), "=>",
    if operator.isCallable(data):
        print "CALLABLE",
    if operator.isMappingType(data):
        print "MAPPING",
    if operator.isNumberType(data):
        print "NUMBER",
    if operator.isSequenceType(data):
        print "SEQUENCE",
    print
        
dump(0)
dump("string")
dump("string"[0])
dump([1, 2, 3])
dump((1, 2, 3))
dump({"a": 1})
dump(len) # function
dump(UserList) # module
dump(UserList.UserList) # class
dump(UserList.UserList()) # instance

<type 'int'> => NUMBER
<type 'string'> => SEQUENCE
<type 'string'> => SEQUENCE
<type 'list'> => SEQUENCE
<type 'tuple'> => SEQUENCE
<type 'dictionary'> => MAPPING
<type 'builtin_function_or_method'> => CALLABLE
<type 'module'> =>
<type 'class'> => CALLABLE
<type 'instance'> => MAPPING NUMBER SEQUENCE
Note that the operator
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The copy Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The copy module contains two functions that are used to copy objects, as shown in Example 1-64.
copy(object) => object creates a "shallow" copy of the given object. In this context, shallow means that the object itself is copied, but if the object is a container, the members will still refer to the original member objects.
Example 1-64. Using the copy Module to Copy Objects
File: copy-example-1.py

import copy

a = [[1],[2],[3]]
b = copy.copy(a)

print "before", "=>"
print a
print b

# modify original
a[0][0] = 0
a[1] = None

print "after", "=>"
print a
print b

before =>
[[1], [2], [3]]
[[1], [2], [3]]
after =>
[[0], None, [3]]
[[0], [2], [3]]
You can also make shallow copies of lists using the [:] syntax (full slice), and you can make copies of dictionaries using the copy method.
In contrast, deepcopy(object) => object creates a "deep" copy of the given object, as shown in Example 1-65. If the object is a container, all members are copied as well, recursively.
Example 1-65. Using the copy Module to Copy Collections
File: copy-example-2.py

import copy

a = [[1],[2],[3]]
b = copy.deepcopy(a)

print "before", "=>"
print a
print b

# modify original
a[0][0] = 0
a[1] = None

print "after", "=>"
print a
print b

before =>
[[1], [2], [3]]
[[1], [2], [3]]
after =>
[[0], None, [3]]
[[1], [2], [3]]
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The sys Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The sys module provides a number of functions and variables that can be used to manipulate different parts of the Python runtime environment.
The argv list contains the arguments that were passed to the script, when the interpreter was started, as shown in Example 1-66. The first item contains the name of the script itself.
Example 1-66. Using the sys Module to Get Script Arguments
File: sys-argv-example-1.py

import sys

print "script name is", sys.argv[0]

if len(sys.argv) > 1:
    print "there are", len(sys.argv)-1, "arguments:"
    for arg in sys.argv[1:]:
        print arg
else:
    print "there are no arguments!"

script name is sys-argv-example-1.py
there are no arguments!
If you read the script from standard input (like "python < sys-argv-example-1.py"), the script name is set to an empty string. If you pass in the program as a string (using the -c option), the script name is set to "-c."
The path list contains a list of directory names in which Python looks for extension modules (Python source modules, compiled modules, or binary extensions). When you start Python, this list is initialized from a mixture of built-in rules, the contents of the PYTHONPATH environment variable, and the registry contents (on Windows). But since it's an ordinary list, you can also manipulate it from within the program, as Example 1-67 shows.
Example 1-67. Using the sys Module to Manipulate the Module Search Path
File: sys-path-example-1.py

import sys

print "path has", len(sys.path), "members"

# add the sample directory to the path
sys.path.insert(0, "samples")
import sample

# nuke the path
sys.path = []
import random # oops!

path has 7 members
this is the sample module!
Traceback (innermost last):
  File "sys-path-example-1.py", line 11, in ?
    import random # oops!
ImportError: No module named random
Example 1-68 demonstrates the builtin_module_names list, which contains the names of all modules built into the Python interpreter.
Example 1-68. Using the sys Module to Find Built-in Modules
File: sys-builtin-module-names-example-1.py

import sys

def dump(module):
    print module, "=>",
    if module in sys.builtin_module_names:
        print "<BUILTIN>"
    else:
        module = _ _import_ _(module)
        print module._ _file_ _

dump("os")
dump("sys")
dump("string")
dump("strop")
dump("zlib")
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The atexit Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
(2.0 only) The atexit module allows you to register one or more functions that are called when the interpreter is terminated.
To register a function, simply call the register function, as shown in Example 1-78. You can also add one or more extra arguments, which are passed as arguments to the exit function.
Example 1-78. Using the atexit Module
File: atexit-example-1.py

import atexit

def exit(*args):
    print "exit", args

# register two exit handler
atexit.register(exit)
atexit.register(exit, 1)
atexit.register(exit, "hello", "world")

exit ('hello', 'world')
exit (1,)
exit ()
This module is a straightforward wrapper for the sys.exitfunc hook.
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The time Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The time module provides a number of functions that deal with dates and the time within a day. It's a thin layer on top of the C runtime library.
A given date and time can either be represented as a floating-point value (the number of seconds since a reference date, usually January 1, 1970), or as a time tuple.
Example 1-79 shows how you can use the time module to get the current time.
Example 1-79. Using the time Module to Get the Current Time
File: time-example-1.py

import time

now = time.time()

print now, "seconds since", time.gmtime(0)[:6]
print
print "or in other words:"
print "- local time:", time.localtime(now)
print "- utc:", time.gmtime(now)

937758359.77 seconds since (1970, 1, 1, 0, 0, 0)

or in other words:
- local time: (1999, 9, 19, 18, 25, 59, 6, 262, 1)
- utc: (1999, 9, 19, 16, 25, 59, 6, 262, 0)
The tuple returned by localtime and gmtime contains the year, month, day, hour, minute, second, day of the week, day of the year, daylight savings flag. The year number is four digits, the day of week begins with 0 for Monday, and January 1 is day number 1.
You can of course use standard string-formatting operators to convert a time tuple to a string, but the time module also provides a number of standard conversion functions, as Example 1-80 illustrates.
Example 1-80. Using the time Module to Format Dates and Times
File: time-example-2.py

import time

now = time.localtime(time.time())

print time.asctime(now)
print time.strftime("%y/%m/%d %H:%M", now)
print time.strftime("%a %b %d", now)
print time.strftime("%c", now)
print time.strftime("%I %p", now)
print time.strftime("%Y-%m-%d %H:%M:%S %Z", now)

# do it by hand...
year, month, day, hour, minute, second, weekday, yearday, daylight = now
print "%04d-%02d-%02d" % (year, month, day)
print "%02d:%02d:%02d" % (hour, minute, second)
print ("MON", "TUE", "WED", "THU", "FRI", "SAT", "SUN")[weekday], yearday

Sun Oct 10 21:39:24 1999
99/10/10 21:39
Sun Oct 10
Sun Oct 10 21:39:24 1999
09 PM
1999-10-10 21:39:24 CEST
1999-10-10
21:39:24
SUN 283
On some platforms, the
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The types Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The types module contains type objects for all object types defined by the standard interpreter, as Example 1-86 demonstrates. All objects of the same type share a single type object. You can use is to test if an object has a given type.
Example 1-86. Using the types Module
File: types-example-1.py

import types

def check(object):
    print object,

    if type(object) is types.IntType:
        print "INTEGER",
    if type(object) is types.FloatType:
        print "FLOAT",
    if type(object) is types.StringType:
        print "STRING",
    if type(object) is types.ClassType:
        print "CLASS",
    if type(object) is types.InstanceType:
        print "INSTANCE",
    print

check(0)
check(0.0)
check("0")

class A:
    pass

class B:
    pass

check(A)
check(B)

a = A()
b = B()

check(a)
check(b)

0 INTEGER
0.0 FLOAT
0 STRING
A CLASS
B CLASS
<A instance at 796960> INSTANCE
<B instance at 796990> INSTANCE
Note that all classes have the same type, as do all instances. To test what class hierarchy a class or an instance belongs to, use the built-in issubclass and isinstance functions.
The types module destroys the current exception state when it is first imported. In other words, don't import it (or any module that imports it!) from within an exception handler.
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The gc Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
(Optional, 2.0 and later) The gc module provides an interface to the built-in cyclic garbage collector.
Python uses reference counting to keep track of when to get rid of objects; as soon as the last reference to an object goes away, the object is destroyed.
Starting with Version 2.0, Python also provides a cyclic garbage collector, which runs at regular intervals. This collector looks for data structures that point to themselves, and attempts to break the cycles. Example 1-87 shows this.
You can use the gc.collect function to force full collection. This function returns the number of objects destroyed by the collector.
Example 1-87. Using the gc Module to Collect Cyclic Garbage
File: gc-example-1.py

import gc

# create a simple object that links to itself
class Node:

    def _ _init_ _(self, name):
        self.name = name
        self.parent = None
        self.children = []

    def addchild(self, node):
        node.parent = self
        self.children.append(node)

    def _ _repr_ _(self):
        return "<Node %s at %x>" % (repr(self.name), id(self))

# set up a self-referencing structure
root = Node("monty")

root.addchild(Node("eric"))
root.addchild(Node("john"))
root.addchild(Node("michael"))

# remove our only reference
del root

print gc.collect(), "unreachable objects"
print gc.collect(), "unreachable objects"

12 unreachable objects
0 unreachable objects
If you're sure that your program doesn't create any self-referencing data structures, you can use the gc.disable function to disable collection. After calling this function, Python 2.0 works exactly like 1.5.2 and earlier.
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
Chapter 2: More Standard Modules
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
"Now, imagine that your friend kept complaining that she didn't want to visit you since she found it too hard to climb up the drain pipe, and you kept telling her to use the friggin' stairs like everyone else..."
—eff-bot, June 1998
This chapter describes a number of modules that are used in many Python programs. It's perfectly possible to write large Python programs without using them, but they can help you save a lot of time and effort.
The fileinput module makes it easy to write different kinds of text filters. This module provides a wrapper class, which lets you use a simple for-in statement to loop over the contents of one or more text files.
The StringIO module (and the cStringIO variant) implements an in-memory file object. You can use StringIO objects in many places where Python expects an ordinary file object.
UserDict, UserList, and UserString are thin wrappers on top of the corresponding built-in types. Unlike the built-in types, these wrappers can be subclassed. This can come in handy if you need a class that works almost like a built-in type, but has one or more extra methods.
The random module provides a number of different random number generators. The whrandom module is similar, but it also allows you to create multiple generator objects.
The md5 and sha modules are used to calculate cryptographically strong message signatures (so-called "message digests").
The crypt module implements a DES-style one-way encryption. This module is usually only available on Unix systems.
The rotor module provides simple two-way encryption.
The fileinput module allows you to loop over the contents of one or more text files, as shown in Example 2-1.
Example 2-1. Using the fileinput Module to Loop Over a Text File
File: fileinput-example-1.py

import fileinput
import sys

for line in fileinput.input("samples/sample.txt"):
    sys.stdout.write("-> ")
    sys.stdout.write(line)

-> We will perhaps eventually be writing only small
-> modules which are identified by name as they are
-> used to build larger ones, so that devices like
-> indentation, rather than delimiters, might become
-> feasible for expressing local structure in the
-> source language.
->      -- Donald E. Knuth, December 1974
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
Overview
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
This chapter describes a number of modules that are used in many Python programs. It's perfectly possible to write large Python programs without using them, but they can help you save a lot of time and effort.
The fileinput module makes it easy to write different kinds of text filters. This module provides a wrapper class, which lets you use a simple for-in statement to loop over the contents of one or more text files.
The StringIO module (and the cStringIO variant) implements an in-memory file object. You can use StringIO objects in many places where Python expects an ordinary file object.
UserDict, UserList, and UserString are thin wrappers on top of the corresponding built-in types. Unlike the built-in types, these wrappers can be subclassed. This can come in handy if you need a class that works almost like a built-in type, but has one or more extra methods.
The random module provides a number of different random number generators. The whrandom module is similar, but it also allows you to create multiple generator objects.
The md5 and sha modules are used to calculate cryptographically strong message signatures (so-called "message digests").
The crypt module implements a DES-style one-way encryption. This module is usually only available on Unix systems.
The rotor module provides simple two-way encryption.
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The fileinput Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The fileinput module allows you to loop over the contents of one or more text files, as shown in Example 2-1.
Example 2-1. Using the fileinput Module to Loop Over a Text File
File: fileinput-example-1.py

import fileinput
import sys

for line in fileinput.input("samples/sample.txt"):
    sys.stdout.write("-> ")
    sys.stdout.write(line)

-> We will perhaps eventually be writing only small
-> modules which are identified by name as they are
-> used to build larger ones, so that devices like
-> indentation, rather than delimiters, might become
-> feasible for expressing local structure in the
-> source language.
->      -- Donald E. Knuth, December 1974
The fileinput module also allows you to get metainformation about the current line. This includes isfirstline, filename, and lineno, as Example 2-2 shows.
Example 2-2. Using the fileinput Module to Process Multiple Files
File: fileinput-example-2.py

import fileinput
import glob
import string, sys

for line in fileinput.input(glob.glob("samples/*.txt")):
    if fileinput.isfirstline(): # first in a file?
        sys.stderr.write("-- reading %s --\n" % fileinput.filename())
    sys.stdout.write(str(fileinput.lineno()) + " " + string.upper(line))

-- reading samples\sample.txt --
1 WE WILL PERHAPS EVENTUALLY BE WRITING ONLY SMALL
2 MODULES WHICH ARE IDENTIFIED BY NAME AS THEY ARE
3 USED TO BUILD LARGER ONES, SO THAT DEVICES LIKE
4 INDENTATION, RATHER THAN DELIMITERS, MIGHT BECOME
5 FEASIBLE FOR EXPRESSING LOCAL STRUCTURE IN THE
6 SOURCE LANGUAGE.
7    -- DONALD E. KNUTH, DECEMBER 1974
Processing text files in place is also easy. Just call the input function with the inplace keyword argument set to 1, and the module takes care of the rest. Example 2-3 demonstrates this.
Example 2-3. Using the fileinput Module to Convert CRLF to LF
File: fileinput-example-3.py

import fileinput, sys

for line in fileinput.input(inplace=1):
    # convert Windows/DOS text files to Unix files
    if line[-2:] == "\r\n":
        line = line[:-2] + "\n"
    sys.stdout.write(line)
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The shutil Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The shutil utility module contains some functions for copying files and directories. The copy function used in Example 2-4 copies a file in pretty much the same way as the Unix cp command.
Example 2-4. Using the shutil Module to Copy Files
File: shutil-example-1.py

import shutil
import os

for file in os.listdir("."):
    if os.path.splitext(file)[1] == ".py":
        print file
        shutil.copy(file, os.path.join("backup", file))

aifc-example-1.py
anydbm-example-1.py
array-example-1.py
...
The copytree function copies an entire directory tree (same as cp -r), and rmtree removes an entire tree (same as rm -r). These functions are illustrated in Example 2-5.
Example 2-5. Using the shutil Module to Copy and Remove Directory Trees
File: shutil-example-2.py

import shutil
import os

SOURCE = "samples"
BACKUP = "samples-bak"

# create a backup directory
shutil.copytree(SOURCE, BACKUP)

print os.listdir(BACKUP)

# remove it
shutil.rmtree(BACKUP)

print os.listdir(BACKUP)

['sample.wav', 'sample.jpg', 'sample.au', 'sample.msg', 'sample.tgz',
...
Traceback (most recent call last):
 File "shutil-example-2.py", line 17, in ?
   print os.listdir(BACKUP)
os.error: No such file or directory
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The tempfile Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The tempfile module in Example 2-6 allows you to quickly come up with unique names to use for temporary files.
Example 2-6. Using the tempfile Module to Create Filenames for Temporary Files
File: tempfile-example-1.py

import tempfile
import os

tempfile = tempfile.mktemp()

print "tempfile", "=>", tempfile

file = open(tempfile, "w+b")
file.write("*" * 1000)
file.seek(0)
print len(file.read()), "bytes"
file.close()

try:
    # must remove file when done
    os.remove(tempfile)
except OSError:
    pass

tempfile => C:\TEMP\~160-1
1000 bytes
The TemporaryFile function picks a suitable name and opens the file, as shown in Example 2-7. It also makes sure that the file is removed when it's closed. (On Unix, you can remove an open file and have it disappear when the file is closed. On other platforms, this is done via a special wrapper class.)
Example 2-7. Using the tempfile Module to Open Temporary Files
File: tempfile-example-2.py

import tempfile

file = tempfile.TemporaryFile()

for i in range(100):
    file.write("*" * 100)

file.close() # removes the file!
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The StringIO Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The stringio module shown in Example 2-8 implements an in-memory file object. This object can be used as input or output to most functions that expect a standard file object.
Example 2-8. Using the StringIO Module to Read from a Static File
File: stringio-example-1.py

import StringIO

MESSAGE = "That man is depriving a village somewhere of a computer scientist."

file = StringIO.StringIO(MESSAGE)

print file.read()

That man is depriving a village somewhere of a computer scientist.
The StringIO class implements memory file versions of all methods available for built-in file objects, plus a getvalue method that returns the internal string value. Example 2-9 demonstrates this method.
Example 2-9. Using the StringIO Module to Write to a Memory File
File: stringio-example-2.py

import StringIO

file = StringIO.StringIO()
file.write("This man is no ordinary man. ")
file.write("This is Mr. F. G. Superman.")

print file.getvalue()

This man is no ordinary man. This is Mr. F. G. Superman.
StringIO can be used to capture redirected output from the Python interpreter, as shown in Example 2-10.
Example 2-10. Using the StringIO Module to Capture Output
File: stringio-example-3.py

import StringIO
import string, sys

stdout = sys.stdout

sys.stdout = file = StringIO.StringIO()

print """
According to Gbaya folktales, trickery and guile
are the best ways to defeat the python, king of
snakes, which was hatched from a dragon at the
world's start. -- National Geographic, May 1997
"""

sys.stdout = stdout

print string.upper(file.getvalue())

ACCORDING TO GBAYA FOLKTALES, TRICKERY AND GUILE
ARE THE BEST WAYS TO DEFEAT THE PYTHON, KING OF
SNAKES, WHICH WAS HATCHED FROM A DRAGON AT THE
WORLD'S START. -- NATIONAL GEOGRAPHIC, MAY 1997
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The cStringIO Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The cStringIO is an optional module, which contains a faster implementation of the StringIO module. It works exactly like the StringIO module, but it cannot be subclassed. Example 2-11 shows how cStringIO is used.
Example 2-11. Using the cStringIO Module
File: cstringio-example-1.py

import cStringIO

MESSAGE = "That man is depriving a village somewhere of a computer scientist."

file = cStringIO.StringIO(MESSAGE)

print file.read()

That man is depriving a village somewhere of a computer scientist.
To make your code as fast as possible, but also robust enough to run on older Python installations, you can fall back on the StringIO module if cStringIO is not available, as Example 2-12 does.
Example 2-12. Falling Back on the StringIO Module
File: cstringio-example-2.py

try:
    import cStringIO
    StringIO = cStringIO
except ImportError:
    import StringIO

print StringIO

<module  'StringIO' (built-in)>
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The mmap Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
(New in 2.0) The mmap module provides an interface to the operating system's memory mapping functions, as shown in Example 2-13. The mapped region behaves like a string object, but data is read directly from the file.
Example 2-13. Using the mmap Module
File: mmap-example-1.py

import mmap
import os

filename = "samples/sample.txt"

file = open(filename, "r+")
size = os.path.getsize(filename)

data = mmap.mmap(file.fileno(), size)

# basics
print data
print len(data), size

# use slicing to read from the file
print repr(data[:10]), repr(data[:10])

# or use the standard file interface
print repr(data.read(10)), repr(data.read(10))

<mmap object at 008A2A10>
302 302
'We will pe' 'We will pe'
'We will pe' 'rhaps even'
Under Windows, the file must currently be opened for both reading and writing (r+, or w+), or the mmap call will fail.
Example 2-14 shows that memory mapped regions can be used instead of ordinary strings in many places, including regular expressions and many string operations.
Example 2-14. Using String Functions and Regular Expressions on a Mapped Region
File: mmap-example-2.py

import mmap
import os, string, re

def mapfile(filename):
    file = open(filename, "r+")
    size = os.path.getsize(filename)
    return mmap.mmap(file.fileno(), size)

data = mapfile("samples/sample.txt")

# search
index = data.find("small")
print index, repr(data[index-5:index+15])

# regular expressions work too!
m = re.search("small", data)
print m.start(), m.group()

43 'only small\015\012modules '
43 small
Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!
The UserDict Module
Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter
The UserDict module contains