Snowpipe Streaming Hits 41 GB/s Benchmark with Elastic Channels

Snowpipe Streaming Hits 41 GB/s Benchmark with Elastic Channels

Skip to content

Blog / Data Engineering / Achieving 41 GB/sec on 1 Table: Benchmarking Elastic Channels for Snowpipe Streaming

Oct 19, 2026 / 14 min read Data Engineering Copy post link Open in Claude Open in ChatGPT

Achieving 41 GB/sec on 1 Table: Benchmarking Elastic Channels for Snowpipe Streaming

Brad Culberson

Legend has it that Snowflake got its name because the company’s founders just really loved hitting the slopes, so it only felt right to test out our new Elastic Channels by putting an entire ski season's worth of resort sales through the system.

As your applications grow and more devices come online, you need a simple way to bring data into Snowflake for analytics and AI. We recently announced the general availability of Elastic Channels for Snowpipe Streaming , which let you send events directly from applications, devices and services while Snowflake manages ingestion scale. But how does that simpler path perform as data volumes grow?

To explore that question, we created a synthetic resort-sales workload and increased the number of independent producers achieving a total throughput of 41 GB/sec. We ran this benchmark with the destination as a Snowflake FDN (native) table and repeated it for the Snowflake managed Apache Iceberg™ table. This post examines how throughput and data freshness changed as the load increased and what the results mean for your own workloads.

Background

Snowpipe Streaming delivers rigorous exactly-once semantics and strict message ordering. For mission-critical workloads such as change data capture (CDC), financial transactions and inventory management, that meticulous bookkeeping is nonnegotiable — complete with explicit channel management, offset tracking and client-side schema coordination.

But if you’re building telemetry pipelines, Internet of Things (IoT) device networks or high-volume log analytics where at-least-once delivery is entirely sufficient, all that extra overhead can become unnecessary friction. That’s precisely why we introduced Elastic Channels: to strip away the complexity of channel lifecycle management and offset bookkeeping. By letting Snowflake handle the scaling and channel distribution automatically, you get lightning-fast ingestion for your lighter-weight data streams without the engineering overhead.

The previous version of Snowpipe Streaming had clients generate columnar data and metadata for every batch of files, upload it and register it with Snowflake for processing. In 2025, Snowflake released the high-performance architecture : server-side PIPE objects, Rust-based SDK cores and a redesigned ingest path that removed much of the client-side burden. The high-performance architecture changed the client so now it only needs to generate and send JSON payloads to Snowflake. Snowflake accepts those JSON payloads and asynchronously processes this data with PIPEs running in our highly scalable Snowpipe infrastructure. These changes removed a substantial amount of the compute and memory overhead from the clients, simplifying deployments. Elastic Channels ( client.get_elastic_channel() ) build on the high-performance architecture — one SDK client opens a pooled channel automatically, with no channel names and no offset management — while writing into the same high-performance pipe.

The question we set out to answer in this test was straightforward: How fast can Elastic Channels actually go, and how efficiently?

The answer surprised us. With an optimal client-side design and a horizontally scaled Kubernetes fleet, we measured sustained aggregate throughput more than 3.5x previous versions , on fewer total cores, less memory and far fewer worker processes than comparable petabyte-scale ingest benchmarks published elsewhere in the industry.

At a glance

Snowpipe Streaming with Elastic Channels sustained 41.8 GB/sec (133.5 million rows per second) to a single FDN Snowflake table — more than 3.5x previous versions of Snowpipe Streaming — using 455 lightweight Kubernetes workers on 455 vCPU and 455 GiB of memory, with zero errors and linear scaling across every step tested.

Peak throughput (uncompressed)

41.8 GB/sec

Peak row rate

133.5 million rows per second

Workers

455 (1 elastic channel)

Compute footprint

455 vCPU · 455 GiB memory

Kubernetes nodes

15 × m7g.8xlarge (Graviton)

vs. previous Snowpipe Streaming

>3.5x higher throughput

What we tested

We built an open source benchmark harness ( snowflake-elastic-channels ) that exercises the snowpipe-streaming Python SDK directly:

Simple deployment: One worker pod = one SDK client = one Elastic Channel against a shared RESORT_SALES_PIPE.

Rate and throughput configuration: Target rate per worker for a fixed duration; aggregate throughput is the sum across pods.

Live observability: A cluster-resident monitor tails worker logs and writes throughput/latency windows to Snowflake ( ELASTIC_K8S_METRICS ) while the run is still in progress.

Typed landing: MATCH_BY_COLUMN_NAME = CASE_SENSITIVE maps JSON keys straight to table columns (not a single VARIANT blob).

Configurable destination: Data lands in FDN tables or Apache Iceberg tables.

Every measurement below is an actual number from a horizontal scale-out on Amazon EKS in us-west-2 , in September 2026. We report uncompressed bytes and row counts as observed by the SDK and monitor — no extrapolation.

The workload: Synthetic resort sales at production scale

Reading from Kafka, S3 or a public data set introduces producer limits that obscure what Snowpipe Streaming itself can do. So we fabricated data in process to remove any external bottlenecks.

The use case we chose included point-of-sale (POS) transactions across a synthetic ski resort portfolio: 42 resorts, four product categories (season tickets, day tickets, food and drink, merchandise), payment methods, customer types and per-resort pricing tiers. Here is the model’s single-table schema:

TXID, RESORT, CATEGORY, ITEM, CUSTOMER_TYPE, PAYMENT_METHOD, QUANTITY, UNIT_PRICE, AMOUNT, SALE_TIME, RUN_ID, WORKER_ID

Each resort has a deterministic sales profile (relative volume, category mix, price tier, passholder share) so aggregate analytics look realistic — revenue by resort, mix shifts, geographic patterns — rather than like uniform random noise. A Streamlit dashboard in the repo visualizes the live feed as if you were watching a national resort chain on a busy holiday weekend.

We...