Introducing Feature Views

Introducing Feature Views | Databricks Blog Skip to main content

Summary

What it is: Feature Views are a managed framework for defining an ML feature once and using it everywhere — the same definition powers historical data for experimentation and training, and production pipelines for batch or real-time inference.

The challenge it solves: Productionizing real-time ML. Experiment with features in a notebook, and rapidly productionize them with a few API calls. Eliminate training/serving skew, duplicated feature code, and fragile self-managed streaming and online store infrastructure that make ML hard to scale.

The outcome: Features become governed Unity Catalog objects, materialized by managed pipelines, with streaming features served at a 200ms end-to-end p99.

In a perfect world, ML Features are built only once. But for many teams, a feature that works in a notebook still turns into duplicated logic, fragile pipelines, one-off backfills, online store plumbing, and governance overhead. For real-time use cases like fraud detection, personalization, and recommendations, that complexity gets even harder to absorb because models depend on fresh signals to make accurate predictions. Common challenges include: Re-implementing feature logic across real-time and historic training Training/serving skew degrading model performance Discovering and re-using features across use cases Backfilling features with large historic lookback into the online store Maintaining complex production infrastructure at scale Governing and tracking lineage across components and pipelines

Databricks is excited to announce the Public Preview of Feature Views, a framework for creating managed feature pipelines directly within Databricks. With Feature Views, you author a feature once and let the platform handle everything from experimentation to real-time serving. What are Feature Views? A Feature View is a simple, powerful abstraction that spans the full ML lifecycle. A data scientist or ML engineer defines their feature logic — the source, the entity, the time-series column, and the computation. From that one definition, Databrick’s Feature Store generates historical, point-in-time-accurate data for experimentation and training. When they're ready, users  materialize the Feature View, and Databricks runs the pipelines that compute feature data for efficient inference. The  same Feature definition supports both batch and streaming sources. Experimentation and productionization are the same for both sources. Switching from a batch source to a streaming source is as simple as a few lines of code. Here's the same feature view definition, running as a streaming and a batch feature. Why Feature Views? 1. One definition, no skew The single biggest source of failure in real-time ML is the gap between how a feature is computed for training and how it's computed at serving time. Feature Views close that gap by construction: there is a single definition, and the platform computes the training values and the online inference values against that single definition so they match. For ML teams, this means much less code to maintain and a much smoother path to production. Better recommendations for hundreds of millions of travelers start with better features. Feature Views cut our feature code dramatically - our data scientists go faster and focus on what drives traveler value, not how to compute it. —Jules Marshall, Sr. Director of Data, Skyscanner 2. Genie Code for Experimentation Get building quickly and easily with the Feature Engineering Client SDK and Genie Code. The SDK makes it simple to declare features locally in a notebook, instantly compute them correctly over historical data, and assemble a point-in-time-accurate training set. Because Databricks co-locates feature definitions, feature data, model training, and MLflow in one environment, data scientists can move from feature idea to model experiment in a single notebook. With Genie Code, teams can use Feature Views to run one-shot model-experimentation workflows: identifying the right data sources, generating feature ideas, and experimenting with models and data in a single notebook. 3. Production-ready pipelines you don't have to operate When a feature is ready for production, register it in Unity Catalog and call materialize_features. Databricks creates and manages the pipelines on your behalf, writing to the appropriate online and offline stores. Production-ready means high-quality data, scalable infrastructure, and mission-critical reliability. Feature Views orchestrates battle-tested GA products like Lakebase and RTM under the hood, optimizing how components work together to support Feature Serving workloads. Corner cases work out of the box, such as backfilling long windows, stream features, or expiring stale rows from the online store.

4. Real-time freshness when you need it For use cases where every new event should immediately change the value served to the model, Feature Views support streaming features sourced from Kafka, delivering end-to-end p99 latency of 200ms from event to online availability. A RollingWindow looks backward from each event's timestamp with millisecond resolution, so an aggregate like "sum of transactions in the last 10 minutes" is always current.

Under the hood, Databricks orchestrates the components that make this fast: Spark Realtime Mode processes events continuously and updates rolling aggregates per event rather than waiting for microbatches; Lakebase serves as a streaming-optimized online store that minimizes write amplification for frequent, small upserts; and Model Serving retrieves features at inference time. You author the rolling-window feature — the platform builds the pipeline. 5. Governed in Unity Catalog, integrated across the platform Materialized Features are data, and they should be governed like data. In Databricks, Feature Views are first-class Unity Catalog objects — discoverable, access-controlled, and tracked with full lineage. Features are packaged with the model: when you log a model with MLflow, its feature dependencies are recorded, and at inference time, Model Serving automatically looks up the required features — no custom lookup code, no manual plumbing. Combined with MLflow, Model Serving, and Genie Code, Feature Views make Databricks a single place to develop, deploy, and govern your entire ML stack. Genie Code is natively integrated with Feature Views, so data...