Polars is fast because it has an “Optimizer” (like a SQL database) that rewrites your code before running it. But sometimes, queries are still slow. To fix them, you need to see what the optimizer is doing.
⚡ Quick Fix: Polars Query Profiling with explain() and profile()
q.explain() shows the optimized execution plan before any data moves — check that SELECTION is nested inside Csv SCAN to confirm predicate pushdown fired. q.profile() runs the query and returns per-node timing so you can pinpoint exactly which step is burning the most time.
import polars as pl
q = (
pl.scan_csv("data.csv")
.filter(pl.col("sales") > 100)
.group_by("region")
.agg(pl.col("sales").sum())
)
# Check the optimized plan
print(q.explain())
# Get per-node timing
df, profile_df = q.profile()
print(profile_df)A SELECTION sitting above CSV SCAN in the plan is the single most common cause of avoidable slowdowns — the full breakdown below shows how to spot and fix it.
1. describe_plan() (The Blueprint)
This shows you the “Lazy” plan before optimization.
import polars as pl
q = (
pl.scan_csv("data.csv")
.filter(pl.col("sales") > 100)
.group_by("region")
.agg(pl.col("sales").sum())
)
print(q.describe_plan())This prints a tree showing the order of operations you wrote.
2. explain() (The Optimized Plan)
This shows you what Polars will actually run. This is where you check for Predicate Pushdown.
print(q.explain())
Output:
AGGREGATE
[col("sales").sum()] BY [col("region")] FROM
Csv SCAN data.csv
PROJECT */? COLUMNS
SELECTION: [(col("sales")) > (100)]What to look for:
- Read it bottom-up: Polars execution plans print with the final step at the top and the initial data read at the bottom.
- Check the SELECTION (Filter): You want to see your filter (
SELECTION) nestled right next to theCsv SCANat the bottom of the tree. This proves Polars is filtering rows while reading the file from disk (Fast!). - The Red Flag: If your filter appears near the top of the tree (after other major transformations), it means Polars is reading and processing too much data into memory before finally filtering it out.
3. profile() (The Stopwatch)
If you want to know exactly how long each step took, use profile() instead of collect(). (Note: Requires executing the query, so it takes time).
# Run the query and record timing df, profile_df = q.profile() print(profile_df)
Output:
┌───────────────────────────┬──────────────┐ │ node ┆ time (µs) │ ╞═══════════════════════════╪══════════════╡ │ AGGREGATE ┆ 1500 │ │ FILTER ┆ 500 │ │ CSV SCAN ┆ 20000 │ └───────────────────────────┴──────────────┘
If “CSV SCAN” is huge, your disk is the bottleneck. If “AGGREGATE” is huge, your CPU is the bottleneck.
Polars explain() and profile() Show You Exactly Where Your Query Loses Time
describe_plan() reflects the order you wrote the query; explain() reflects what Polars actually runs — always read explain() output, not describe_plan(), when diagnosing a slow query. Predicate pushdown is the optimization most worth verifying:if SELECTION appears at the very top of the explain() tree instead of nested inside Csv SCAN, Polars reads every row before filtering, and the fix is restructuring the query so the filter comes directly after scan_csv() with no intervening transformation that blocks pushdown. For profile() output, address the slowest node first — a dominant CSV SCAN time points to a format problem (switch from CSV to Parquet), while a dominant AGGREGATE time points to a cardinality problem (too many unique groups for available CPU cores, solved by pre-filtering before the group_by).
