Python Cookbook

Reading Lines with Continuation Characters

Credit: Alex Martelli

Problem

You have a file that includes long logical lines split over two or more physical lines, with backslashes to indicate that a continuation line follows. You want to process a sequence of logical lines, rejoining those split lines.

Solution

As usual, a class is the right way to wrap this functionality in Python 2.1:

class LogicalLines:

    def _ _init_ _(self, fileobj):

        # Ensure that we get a line-reading sequence in the best way possible:
        import xreadlines
        try:
            # Check if the file-like object has an xreadlines method
            self.seq = fileobj.xreadlines(  )
        except AttributeError:
            # No, so fall back to the xreadlines module's implementation
            self.seq = xreadlines.xreadlines(fileobj)

        self.phys_num = 0  # current index into self.seq (physical line number)
        self.logi_num = 0  # current index into self (logical line number)

    def _ _getitem_ _(self, index):
        if index != self.logi_num:
            raise TypeError, "Only sequential access supported"
        self.logi_num += 1
        result = []
        while 1:
            # Intercept IndexError, since we may have a last line to return
            try: 
                # Let's see if there's at least one more line in self.seq
                line = self.seq[self.phys_num]
            except IndexError:
                # self.seq is finished, so break the loop if we have any
                # more data to return; else, reraise the exception, because
                # if we have no further data to return, we're finished too
                if result: break
                else: raise self.phys_num += 1 if line.endswith('\\\n'): result.append(line[:-2]) else: result.append(line) ...