AI Data Engineering: New Smart Pipelines in Snowflake

AI Data Engineering: New Smart Pipelines in Snowflake

Skip to content

Blog / Product and Technology / Data Engineering in the AI Era: New Snowflake Tools Built for Smart Pipelines

JUN 10, 2026 / 10 min read Product and Technology Data Engineering in the AI Era: New Snowflake Tools Built for Smart Pipelines

Abhishek Kashyap +1

AI has made it easier than ever to build. However, easier to build is not the same as built to last. If you have brittle, fragile systems, AI is only going to make it worse, not better. That's why you need a platform built to make the most of AI.

At Snowflake Summit 2026, we announced new capabilities that put our customers at the forefront of data engineering today. We've added AI directly into workflows and made it easier to build data pipelines from start to finish. These new features are designed for every type of data engineer. They work where your data lives: in Snowflake, in open and interoperable lakehouses or both. Whether you write SQL, Python or build ML models, everything you need to construct pipelines exists in one place. With Snowflake, you get elastic compute performance that scales, seamless connectivity to data wherever it lives, and enterprise-grade governance capabilities for secure, trusted data with consistent business context.

Faster time-to-production with AI

Figure 1: Snowflake CoCo outperforms generic coding agents for data engineering tasks. 1

With new agentic workflows, AI operates directly within your local environment to build end-to-end solutions. For real data engineering work, Snowflake CoCo sets the bar for leading coding agents. Benchmarks comparing to Claude Code running on Opus 4.7, for instance, show that CoCo uses 51% fewer tokens and takes 8% fewer steps to get the job done. 2

Bringing context-aware assistance and purpose-built skills for Snowflake data engineering features, CoCo operates within your security perimeter and crucially understands your enterprise data context. With access to the latest models, like Claude Opus 4.8, Claude Sonnet 4.6 and GPT 5.5, data engineers can use it in Snowsight, through the CoCo CLI or now through a new desktop app (public preview). Use prebuilt or custom skills to migrate Spark pipelines , deploy Python code , automate dbt workflows , optimize performance and more — all from a single prompt.

Autonomous pipelines you can trust

Every organization wants AI-ready data delivered continuously, at low latency, from an ever-growing set of sources. The old way, with handcrafted orchestration scripts, brittle incremental logic and manual deployments, is hard to scale. Declarative workflows let you define what you want — and Snowflake handles how it gets done.

Wolt (part of DoorDash) standardized on Apache Iceberg to give us the flexibility to run each workload on the right engine. We use Snowflake Dynamic Iceberg Tables to enrich, prepare and automatically refresh data in our data lake — we define a single query with a target freshness and Snowflake manages the incremental updates and orchestration. With Dynamic Tables on Apache Iceberg, we have launched pipelines faster, cut maintenance time, and reduced the overhead of our incremental pipelines. Raimund Kämmerer

Staff Data Engineer, Wolt

Faster, more flexible Dynamic Tables

Dynamic Tables removes hours of manual effort by automating refreshes to your data based on a defined query and target freshness. Dynamic Tables offers leading performance and low latency for incremental pipelines. At Summit, Sergey Labetsik, a senior data engineer at Wind Creek Hospitality, demonstrated how his team was able to deliver food vouchers to guests within a minute of eligibility. By migrating a dbt batch job to a Dynamic Tables pipeline, they cut end-to-end latency to under a minute, a vast improvement from the 30-minute schedule that the job had been running on.

Figure 2: Benchmarks showing up to 2.8x faster refresh performance on Dynamic Tables.

Snowflake announced a series of updates to native declarative workflows to make them more performant, interoperable and expressible, including:

Faster Dynamic Tables refresh performance (generally available): Accelerate workloads by up to 2.8x in a number of areas, including aggregate functions, qualify/rank (SCD-1), cluster-by operations and joins — all measured on Gen2 warehouses.

Custom incrementalization (public preview): Optimize performance for complex transformations by writing your own refresh logic using MERGE or INSERT statements while retaining all the benefits of Dynamic Tables like automatic scheduling, dependency tracking and replication.

Adaptive refresh (public preview): Automatically determine the most efficient refresh method for each cycle. No tuning required. Snowflake systematically chooses between incremental and reinitializations to optimize for cost, prevent failures on complex queries and eliminate manual tuning.

Dynamic Table materialization in dbt (adapter version 1.11.5): Optimize incremental processing by simply changing materialization type in dbt. Composable with other dbt models in the pipeline.

DCM Projects (public preview): Manage infrastructure declaratively by enabling a way to version, test and deploy diverse transformation pipelines on Snowflake.

Bring dbt into Snowflake natively

With dbt Projects on Snowflake, you can use familiar Snowflake features to create, edit, test, run and manage your dbt Core projects. Deploying a dbt Project object gives you built-in observability, CI/CD integration, and removes the infrastructure overhead associated with managing it yourself.

As early adopters of dbt Projects, we worked hand in hand with Snowflake to help shape the roadmap around how our teams actually build and operate. That allowed a lean team to move faster, while creating a more modular, governed and scalable foundation for analytics and for enabling AI across the group. António Costa

Director of Data Engineering, Aviv Group

With the updates announced at Summit, more customers are standardizing on dbt Projects. They're able to replace the management of dbt Core while gaining access to dbt Fusion and more observability:

dbt Fusion (generally available) is now included as a version with dbt Projects on Snowflake. Provided through our partnership with dbt Labs, any dbt Project can access Fusion, which is designed to improve compilation times for many complex builds.

Enhanced dbt DAG with column-level lineage (generally available) uses…