Polars Structs: How to Pack and Unpack Multiple Columns
In Pandas, you are stuck with flat columns. In Polars, you can put columns inside other columns. This is called a Struct (structure). It’s like…
Extending Polars: How to Write Custom Namespaces (The API for 2026)
As you become an expert in Polars, you will inevitably find yourself reusing the same complex expressions. Fortunately, one highly effective way to avoid endless…
Data Engineering with Polars: Performing Upserts (Merge) into Delta Tables
In Data Engineering, you rarely just “write” files. You usually have a master dataset (e.g., “All Users”), and every day you get a “Daily Update”…
Visualizing Millions of Rows: Polars + Datashader (Big Data Plotting)
If you try to plot 10 million points with Matplotlib or Seaborn, your computer will freeze. It tries to draw 10 million individual circles, which…
Modern Data Lakes: Reading and Writing Delta Tables with Polars
We’ve used Parquet files because they are fast. But what if you write to a Parquet file while someone else is reading it? The file…
Polars Performance Tuning: 3 Tricks to Make Your Code Faster
Polars is fast by default. But you can make it slower if you write “Pandas-style” code. Here are the top 3 optimisations for Polars Performance…
Polars Feature Engineering: Lags, Diffs, and Percent Changes
If you are training a Machine Learning model to predict stock prices or sales, you can’t just feed it “Today’s Price.” You need to feed…
Big Data in Polars: How to Scan and Process Multiple Files at Once
In the real world, data is rarely in one big CSV. It’s usually split, and this is where you might want to use Polars to…
Big Data in Polars: Reading and Writing Partitioned Parquet Files
When you have 1TB of data, you don’t save it in one giant file. You split it up. Polars Partitioned Parquet is handling large datasets…
Processing Data Larger Than RAM: The Polars Streaming Engine (sink_parquet)
In Pandas, if your dataset is bigger than your RAM (e.g., a 30GB file on a 16GB laptop), you get a MemoryError. The Polars Streaming…
