Reading parquet files
Apache Parquet is another file format that makes use of columnar compression for efficient read and write operations. It was designed to be compatible with big data ecosystems such as Hadoop and can handle nested data structures and sparsely populated columns. Though the parquet and feather formats share a similar base, parquet has a better compression routine than feather. The compressed file is smaller in parquet than it is in feather. Columns with similar data types use the same encoding for compression. The use of different encoding schemes for the compression of parquet makes it efficient. Just like feather, parquet is a binary file format that can work well with all pandas data types and is supported across several ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access