Real-time feature engineering is the process of turning live events, such as clicks, payments or sensor readings, into model inputs within milliseconds to seconds, so predictions reflect what is happening now. It combines streaming pipelines, a low-latency online store and on-demand transformations computed at request time. In 2026 it powers fraud detection, recommendations, predictive maintenance and, increasingly, the context that AI agents use to act.
Updated October 2026
What changed in 2026
- Feature platforms joined data platforms. Databricks announced that Tecton is joining Databricks to power real-time data for personalised AI agents, a sign that feature stores are now core data infrastructure.
- Streaming engines modernised. Apache Kafka 4.0 (March 2025) runs without ZooKeeper, and Apache Flink 2.0 introduced disaggregated state management for cloud-native stateful streaming.
- Agents need fresh context. The same real-time features that feed fraud models now feed LLM agents through tools and retrieval, so freshness and governance matter beyond classic ML.
- Data readiness is the gating factor. Gartner warns that organisations will abandon 60% of AI projects unsupported by AI-ready data through 2026.
What is a feature, and why does "real time" matter?
A feature is a number or category a model uses to make a prediction: "average order value over 30 days", "transactions in the last 5 minutes", "bearing temperature trend over the last hour". Feature engineering is choosing, building and transforming these inputs so the model can learn useful patterns.
In batch ML, features are computed nightly. That is fine for a churn score. It fails when the signal is minutes old. A card-testing attack, a shopper's current session or a motor starting to overheat all need features that are seconds fresh.
Batch vs streaming vs on-demand features
| Feature type | Computed | Freshness | Example |
|---|---|---|---|
| Batch | Scheduled jobs on a warehouse or lakehouse | Hours to a day | Customer lifetime value |
| Streaming | Continuously from an event stream (Kafka, Kinesis, Pub/Sub) with Flink or Spark | Seconds | Payments per card in last 10 minutes |
| On-demand | At request time inside the serving path | Instant | Distance between this login and the last one |
Most production systems mix all three. Databricks' explainer on real-time features is a good walk-through of these patterns.
The real-time feature architecture
- Event sources: apps, payment systems, PLCs and IoT gateways publish events.
- Stream transport: Kafka or a managed equivalent buffers and orders events.
- Stream processing: Flink, Spark Structured Streaming or a streaming database computes windowed aggregations (counts, sums, averages over sliding windows).
- Online store: a low-latency key-value store (Redis, DynamoDB, Cassandra and similar) holds the latest feature values.
- Offline store: the same features, with timestamps, land in the lakehouse for training.
- Feature registry: definitions, owners and versions live in one place. Open-source Feast supports on-demand feature views alongside batch and stream features.
- Model serving: the model or agent fetches features by entity key and returns a prediction.
At the edge, steps 2 to 4 often run on a gateway near machines. Our guide to edge data management covers how to filter and aggregate data before it leaves a site.
Feature engineering techniques for streaming data
- Windowed aggregations: tumbling, hopping and sliding windows for counts, sums, min/max and rates.
- Time-decay features: exponentially weighted averages that favour recent behaviour.
- Velocity and change features: rate of change in temperature, spend or clicks.
- Sessionisation: grouping events into sessions with inactivity gaps.
- Embeddings: vector representations of users, items or text, updated incrementally and served from a vector store.
- Signal processing for sensors: RMS, kurtosis and FFT bands from vibration data, a staple of predictive maintenance.
- Categorical handling: hashing or learned embeddings instead of one-hot encodings that explode in size.
Common pitfalls and best practices
Training-serving skew
If training features are computed in SQL and serving features in Java, they will drift apart. Define each feature once and generate both offline and online versions from that definition.
Point-in-time correctness
Training data must use feature values as they were at prediction time. Leaking future values gives great offline scores and poor production results. Feature stores handle this with timestamped joins.
Late and out-of-order events
Use event time, watermarks and allowed lateness, not processing time, so a delayed sensor batch does not corrupt aggregates.
Latency budgets
Set an end-to-end budget (for example, under 50 milliseconds for card authorisation) and measure each hop. Precompute heavy aggregations; keep on-demand logic light.
Monitoring and governance
Track feature freshness, null rates and distribution drift, and record which features each model and agent used. Our ML platform tools guide covers monitoring options, and AgentOps explains how this extends to agents.
Where it's heading: 2026 → 2030
| Period | Real-time features look like |
|---|---|
| Around 2020 (history) | Custom streaming code per use case, early feature stores, frequent train-serve skew. |
| 2026 (now) | Feature platforms merge into data platforms; Kafka 4.0 and Flink 2.0 simplify streaming; agents consume the same fresh features. |
| 2028 (expected) | Our view: declarative feature definitions compile to batch, stream and edge targets automatically, with built-in lineage for audits. |
| 2030 (our view) | A unified "context layer" serves features, embeddings and documents to models and agents with consistent freshness and access policy. |
How NexaStack helps
NexaStack runs the inference and agent side of a real-time ML system on your own infrastructure, from GPU clusters to edge gateways. Teams use it to deploy models next to their streaming and feature pipelines, keep latency low, monitor predictions and control which models and agents can read which data.
FAQ
What is real-time feature engineering?
It is computing model input features from live event streams or at request time, so predictions use data that is seconds old rather than hours old.
Do I need a feature store for real-time ML?
Not always, but a feature store or feature platform makes it much easier to avoid training-serving skew, ensure point-in-time correctness and reuse features across teams.
What are some examples of feature engineering?
Counting transactions per card in the last 10 minutes, computing a rolling average temperature, calculating days since last purchase, or creating text embeddings for search.
What are the latest real-time ML trends in 2026?
Feature platforms merging into lakehouse platforms, real-time context for AI agents, cloud-native stateful streaming with Flink 2.0, ZooKeeper-free Kafka and more processing at the edge.
What latency counts as real time?
It depends on the use case. Payment fraud often needs tens of milliseconds end to end, recommendations a few hundred milliseconds, and many industrial alerts a few seconds.
Serve fresh features to models and agents
Building fraud, personalisation or industrial models that need second-fresh data? Book a NexaStack demo to see low-latency inference on your own infrastructure.
