Python Cookbook

Counting Lines in a File

Credit: Luther Blissett

Problem

You need to compute the number of lines in a file.

Solution

The simplest approach, for reasonably sized files, is to read the file as a list of lines so that the count of lines is the length of the list. If the file’s path is in a string bound to the thefilepath variable, that’s just:

count = len(open(thefilepath).readlines(  ))

For a truly huge file, this may be very slow or even fail to work. If you have to worry about humongous files, a loop using the xreadlines method always works:

count = 0
for line in open(thefilepath).xreadlines(  ): count += 1

Here’s a slightly tricky alternative, if the line terminator is '\n' (or has '\n' as a substring, as happens on Windows):

count = 0
thefile = open(thefilepath, 'rb')
while 1:
    buffer = thefile.read(8192*1024)
    if not buffer: break
    count += buffer.count('\n')
thefile.close(  )

Without the 'rb' argument to open, this will work anywhere, but performance may suffer greatly on Windows or Macintosh platforms.

Discussion

If you have an external program that counts a file’s lines, such as wc -l on Unix-like platforms, you can of course choose to use that (e.g., via os.popen( )). However, it’s generally simpler, faster, and more portable to do the line-counting in your program. You can rely on almost all text files having a reasonable size, so that reading the whole file into memory at once is feasible. For all such normal files, the len of the result of readlines gives you the count of lines in ...