Introducing Polars Nanook.

Polars Nanook is our next generation data platform for high performance scalable workloads on cloud and on-premise for either humans or agents.

Aside from the rebrand, we've been able to ship very big features:

- SQL support
- Huge leap in coverage distributed algorithms
- GPU support
- Faster than any Spark accelerator we can find

pola.rs/posts/polars-c…

Working on the next version of Effective Polars.

The book is more than twice as long as the second edition and now includes detailed content on extending Polars in Rust.

I'm excited to get this out.

Polars 2.1 is released!

Three weeks after 2.0 we're shipping a big one. A few highlights:

- Spill to disk for group-by and window functions
- Recursive CTE's in SQL
- Reading and writing encrypted Parquet
- Better memory tracking for out-of-core
- And over 90 performance nk below

First successful run of Polars with dbt 🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉

#OneLake #ApacheIceberg #Polars #dbt #SQL

We ran Polars 2.0 SQL, DuckDB and DataFusion on an @nvidia Vera CPU and compared it to an Intel Xeon 6, m8id.metal-96xl.

On the NVIDIA Vera, Polars ran 2 to 2.5x faster compared to Xeon, and 2 to 3x faster than the runner-up.

pola.rs/posts/nvidia_v…

#polars 2 is very good
djouallah.github.io/lakehouse_benc…

We just released Polars 2.0.

It removed many of our legacy decisions makes the streaming engine our default and promotes SQL to a first class citizen within Polars.

It comes with initial out-of-core (spill to disk) support, a new Map data type and a lot of performance we think Polars is now one of the fastest analytical SQL engines on a single node. See benchmarks in the post: pola.rs/posts/release-…

Polars 2.0.0rc2 has been released.
We expect this to be the last release candidate and that Polars 2.0 will go live next week.

Install now by running `pip install --pre polars`

Link to the full release guide: docs.pola.rs/releases/upgra…

Profile any Polars query to view its progress and optimize performance with one line change:

pl.Config.enable_monitoring()

Read the full blog: pola.rs/posts/profile-…

Come join us at the second day of Pydata Amsterdam!

Learn about our new query profiler and let an agent optimize your queries!

We are happy to announce the first pre-release of Polars 2.0.

Polars 2.0 will bring the streaming engine as a default and improves overall strictness of Polars.

Give it a spin and share your thoughts. We expect the final Polars 2.0 release in 3-4 weeks.

pola.rs/posts/announci…

The Polars GPU engine now runs on a new streaming backend, built with NVIDIA on their RapidsMPF library.

• Queries are no longer bound by GPU memory. Data flows through the query graph in morsels that spill to host memory when device memory gets tight.

• The same query scales onstruct a RayEngine, pass it to .collect(), and there is no cluster to configure and no change to the query itself.

Full post: pola.rs/posts/gpu-stre…

We've released Python Polars 1.44. Some of the highlights:

• Schema evolution when writing to Iceberg
sink_iceberg() gained schema_mode="merge", so appending a frame that has a new column extends the table schema instead of raising. Rows written earlier read back as null for e rate limiting for cloud I/O
Polars now learns how many requests per second S3, GCS, or Azure will actually serve and paces itself to stay under it, so large scans spend less time in retries and are far less likely to fail on throttling.

• Correlated subqueries in SQL
Subqueries that reference the outer query now work in WHERE, in the select list, and in EXISTS and IN.

Blog post: pola.rs/posts/polars-1…
Full changelog: github.com/pola-rs/polars…

As datasets grow, your tools can limit what analysis is possible. That causes teams to slows down, or forces them to work with samples rather than the whole dataset.

BMLL Technologies tackled this issue by migrating their market data workloads to Polars, resulting in a 48x . At that speed, a single on-demand machine can aggregate over 1.5 TB of market data in under 4 minutes, removing the need for overnight batch processing infra.
Work now fits on a single machine, allowing teams to iterate daily instead of weekly, compounding small wins faster and capturing more basis points.

Read the full case study here: pola.rs/posts/case-bml…

You can turn a column into labeled buckets. With if/elif/else logic takes one chained expression: pl.when().then().otherwise().

Conditions are checked top to bottom, and the first match wins, exactly like Python's if/elif/else. Chain as many .when() branches as you need, and orized in Rust rather than row by row.

Note: a bare string inside then() is read as a column name, so .then("adult") looks for a column called "adult" and raises a ColumnNotFoundError. If you want to fill a string value instead, use pl.lit().

We will ship the first Polars 2.0 release candidate next week. Polars 2.0 will be great for casual Polars users as it will default to the streaming engine, meaning your code will be faster without doing anything.

So the first release candidate next week. Give it a spin!

Time series datasets are seldom perfect. Polars gives you a whole menu for repairing them, and the right pick depends on what you need.

`fill_null` accepts more than a constant value to fill. `strategy="forward"` carries the last value forward, the natural choice for state that es, like a stock ticker. `strategy="backward"` pulls the next known value back, and there is also `"mean"`, `"min"`, `"max"`, `"zero"`, `"one"`, plus a `limit` parameter to cap how many consecutive nulls a fill may bridge.

For values that were moving smoothly while a sensor was offline, for example, `interpolate()` draws a straight line between the surrounding points. Combine it with `.over("sensor_id")` and each sensor is filled using only its own history.

When migrating your pandas pipeline to Polars there are several ways to go about it.
Read which one works best for you and the tips and tricks to make it successful here: pola.rs/posts/pandas-t…