Python Polars: The Definitive Guide

Chapter 15. Reshaping

In the last chapter we focused on aggregating data to create informative summaries. However, what should you do if the data is not in the right shape to perform these aggregations? Reshaping data is a crucial step in the data analysis process.

In this chapter, you’ll learn how to:

• Reshape data to make it more suitable for analysis

• Change the dimensions of the data to make it more suitable for analysis, improve computational performance, or prepare it for visualization

• Use the various methods Polars offers, such as df.pivot(), df.unpivot(), df.transpose(), df.explode(), and df.partition_by()

The instructions to get any files you might need are in Chapter 2. We assume that you have the files in the data subdirectory.

Wide Versus Long DataFrames

Wide DataFrames have many columns and few rows. The idea is that every row contains a column with an identifier, and the data is spread over many columns. This format is often used when there are multiple measurements per observation. An example of wide data would be the following:

grades_wide = pl.DataFrame(
    {
        "student": ["Jeroen", "Thijs", "Ritchie"],
        "math": [85, 78, 92],
        "science": [90, 82, 85],
        "history": [88, 80, 87],
    }
)

grades_wide

shape: (3, 4) ┌─────────┬──────┬─────────┬─────────┐ │ student │ math │ science │ history │ │ --- │ --- │ --- │ --- │ │ str │ i64 │ i64 │ i64 │ ╞═════════╪══════╪═════════╪═════════╡ │ Jeroen │ 85 │ 90 │ 88 │ │ Thijs │ 78 │ 82 │ 80 │ │ Ritchie │ 92 │ 85 │ 87 │ └─────────┴──────┴─────────┴─────────┘ ...