Skip to Content
Functional Python Programming - Second Edition
book

Functional Python Programming - Second Edition

by Steven F. Lott
April 2018
Intermediate to advanced content levelIntermediate to advanced
408 pages
10h 42m
English
Packt Publishing
Content preview from Functional Python Programming - Second Edition

Parsing log files – gathering the rows

Here is the first stage in parsing a large number of files: reading each file and producing a simple sequence of lines. As the log files are saved in the .gzip format, we need to open each file with the gzip.open() function instead of the io.open() function or the __builtins__.open() function.

The local_gzip() function reads lines from locally cached files, as shown in the following command snippet:

from typing import Iteratordef local_gzip(pattern: str) -> Iterator[Iterator[str]]:    zip_logs= glob.glob(pattern)
    for zip_file in zip_logs:
        with gzip.open(zip_file, "rb") as log:
            yield (                line.decode('us-ascii').rstrip()                 for line in log)

The preceding function iterates through all files that match the given ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Start your free trial

You might also like

Functional Python Programming - Third Edition

Functional Python Programming - Third Edition

Steven F. Lott

Publisher Resources

ISBN: 9781788627061Supplemental Content