Improving Lakebase Postgres compute cache

Improving Lakebase Postgres compute cache | Databricks Blog Skip to main content

Summary

The standard Postgres cache vs Lakebase Postgres cache

How we created an autoscaling cache that works in tandem with shared buffers and keeps as much data as possible on compute

Production results including 2x throughput, fewer reads from the storage layer and lower latency

The disaggregated storage model of Lakebase Postgres provides a feature rich, flexible and low cost platform. Efficient caching of data is critical to provide high throughput and low latency while data is backed in an object store such as S3. This caching takes place at two layers: in distributed storage, where Postgres pages are materialized for high write throughput and read serving; and on the Postgres compute itself to serve frequently accessed pages from DRAM for ultra fast access. We've been hard at work making improvements to the compute side caching, and in this blog will lay out our near term plans and delve into what has already been shipped to customers. First, some background on how we got here. The standard Postgres cache Databases are famously hungry for DRAM (memory). They primarily use this memory as a data cache and expect access to rows in the cache to be measured in nanoseconds - orders of magnitude faster than even the fastest NVMe drives. Postgres organizes data in rows on pages, and pages actively being accessed must be loaded into a memory area known as "shared buffers". Because Postgres traditionally stores pages using the operating system's filesystem, the OS kernel will also use its flexible page cache to provide caching between Postgres shared buffers and the disk. This shared buffers + page cache scheme works reasonably well but has some downsides and some challenges. Downsides Double buffering which reduces the amount of data you can effectively cache on the compute. Consider a compute with 4 GB of RAM using 1 GB for shared buffers. As you read pages from disk to populate the 1 GB of shared buffers, the reads go through the OS page cache, which also holds that data. You are now consuming 2 GB of RAM to cache 1 GB of data. The OS page cache doesn't know anything about the shared buffers or Postgres internals, so it can't make smart decisions on which pages to replace.

Technical challenges In a disaggregated storage system such as Lakebase Postgres , data read from storage does not travel through the OS filesystem or page cache. Shared buffers is a static parameter, meaning that it is set prior to starting Postgres and cannot be changed without rebooting the database. This is a meaningful challenge for a serverless autoscaling system such as Lakebase. Postgres uses a separate operating system process for each active connection, so the larger the shared buffers - i.e. the more memory you give Postgres - the more memory management the OS must do for each and every connection, which in turn consumes memory.

The Lakebase cache path

Now that we've provided some background, let's talk about how we are solving them at Databricks. Our desired end state is to make the most efficient use of the DRAM on your compute via Postgres dynamic shared buffers that autoscale with your workload and use up to 75% of available memory. We need to eventually adjust our compute platform to leverage autoscaling shared buffers, but we also want to deliver sensible incremental improvements to our customers as they become available. Each incremental delivery allows us to confidently ship one or more pieces of the roadmap while giving real benefit to customers. So even if autoscaling computes are the goal, we started with fixed computes, as covered in the next section. Here’s what we implemented. Larger shared buffers

If you recall from the technical challenges above, a disaggregated system such as Lakebase does not route its reads through the standard OS file system and its page cache. Also recall that Postgres shared buffers are static and cannot autoscale. To solve this we created a layer we called the local file cache (LFC). The LFC acted as a stand-in, creating an autoscaling cache that worked in tandem with shared buffers and kept as much data as possible cached on the compute. This was a clever and pragmatic solution that allowed Lakebase Postgres to launch autoscaling and has been in use on all compute since launch. Although exposed as a single high-speed compute cache to users, the underlying architecture supports up to two tiers: Shared buffers: Postgres's in-memory shared buffer, representing the lowest-latency access path. Local file cache: An expanded secondary cache residing on the compute node's local NVMe, offering higher capacity than memory but requiring disk I/O to access a page.

Shared buffers were tuned conservatively so that they did not consume too much memory when running at minimum configured CU, with the maximum size ever configured at 1 GB of shared buffers and LFC consuming the remainder of the total compute cache capacity (up to 75% of DRAM). Any request that results in a miss across both tiers is routed from the compute node to the distributed storage layer. On larger working sets, capping shared buffers at 1 GB forced most cache hits to pass through the slower LFC tier. The LFC has served us well, but our intent is to retire its current form as we progress towards fully dynamic shared buffers. Note: Fixed computes came first Our first delivery of larger shared buffers targets fixed-size computes, since shared buffers are not yet dynamic. On these, we now disable the LFC and set shared buffers to 75% of DRAM. This is live today for fixed-size computes with CU >= 80. Eliminating the ~1 GB buffer cap keeps hot pages in the fastest memory layer instead of cascading down to local file storage. To see if large shared buffers are enabled for your compute, run show shared_buffers within a Postgres connection. An 80 CU Lakebase endpoint in Databricks should see a value of 20971520 .

Keeping hot data in shared buffers rather than the OS page cache also addresses the downsides described earlier. There is no double buffering, so 1 GB of cached data consumes 1 GB of RAM instead of 2 GB. And because the cache lives inside Postgres rather than the kernel, eviction decisions can be made with knowledge of database state — that positions us to pursue smarter replacement policies than the OS can offer. Sizing shared buffers at 75% of DRAM on fixed-size computes...