Python Polars: The Definitive Guide

Appendix. Accelerating Polars with the GPU

One of Polars’ core philosophies is to make full use of all available processing power on your machine. This includes utilizing all the cores of the CPU, but there’s another type of core we’ve been overlooking: the cores in your graphics card.

NVIDIA graphics cards, for example, use Compute Unified Device Architecture (CUDA) cores. CUDA is a proprietary parallel computing platform that leverages the graphics processing unit (GPU) for accelerated general-purpose computation. Unlike CPUs, which typically have between 2 and 20 cores, a modern GPU can feature over 15,000 cores.

While each individual GPU core is less powerful than a CPU core and is designed for simpler computations, the real advantage comes from its ability to process tasks in parallel. If you can break down complex data processing steps into simpler instructions that can be executed concurrently, the GPU can provide a substantial performance boost.

In this Appendix, we discuss accelerating Polars using the GPU. Specifically, we cover:

• How to install and use the GPU engine

• Supported and unsupported features

• Benchmarks conducted with the GPU engine

• Recommendations for situations where the GPU engine offers the most benefit

• The future of Polars on the GPU

NVIDIA RAPIDS

NVIDIA describes the RAPIDS project as a collection of open source software packages and APIs that give you the ability to execute end-to-end data science and analytics pipelines entirely on NVIDIA ...