Polars is fast by default. But you can make it slower if you write “Pandas-style” code. Here are the top 3 optimisations for Polars Performance Tuning to ensure you are getting maximum speed.
Trick 1: Always Use Lazy Mode (scan_)
We’ve mentioned this before, but it is the #1 rule.
- Bad (Eager):
pl.read_csv("data.csv")loads everything into RAM immediately. - Good (Lazy):
pl.scan_csv("data.csv")allows Polars to see the whole query and optimize it.
# Fast
q = (
pl.scan_csv("data.csv")
.filter(pl.col("sales") > 100)
.group_by("region")
.agg(pl.col("sales").sum())
)
df = q.collect() # Only loads what is neededTrick 2: Filter Early (Pushdown Predicates)
If you are using Lazy mode, Polars does this automatically (“Predicate Pushdown”). But if you are in Eager mode, filter as soon as possible.
Don’t sort or join a 10-million row dataset and then filter it down to 100 rows. Filter it to 100 rows first!
Trick 3: Use Categorical for Strings
If you have a string column with low cardinality (few unique values, like “Country”, “Status”, “Gender”), convert it to Categorical.
String comparisons are slow (checking “United States” vs “United Kingdom” letter by letter). Integer comparisons are fast (checking 1 vs 2).
# Slow String Joins
df = df.with_columns(pl.col("category").cast(pl.Utf8))
# Fast Categorical Joins
df = df.with_columns(pl.col("category").cast(pl.Categorical))This simple cast can speed up groupby and join operations by 5x to 10x.
Key Takeaways
- Polars is fast by default, but writing ‘Pandas-style’ code can slow it down.
- Always use Lazy Mode with
scan_for better performance and optimisation. - Filter early to reduce data size before processing, especially in Eager mode.
- Convert low cardinality string columns to Categorical for faster comparisons.
- These optimisations can significantly enhance Polars performance tuning by improving speed in operations.
