Decoupling Inference from State Updates in Low-Latency Feature Engines via Probabilistic Thinning

2026-06-15

AI Summary

This work addresses the high latency, resource contention, and operational overhead caused by frequent state updates in streaming machine learning. The authors propose a probabilistic sparsification strategy that decouples inference from state persistence: while all events contribute to inference scoring, only those deemed highly informative trigger persistence. This approach enables precise control over the persistence path without requiring high-frequency in-memory control planes or cross-node coordination, while preserving unbiasedness of time-aggregated statistics. By integrating approximate statistics from disk-based key-value stores with variance-aware temporal aggregation modeling, the method reduces persistence events by up to 90%, substantially lowering I/O and serialization costs while maintaining or even improving downstream task performance.

Abstract

Streaming data systems increasingly underpin Machine Learning workflows that maintain large numbers of continuously updated aggregations. In production settings, each incoming event typically triggers read-modify-write operations to persistent storage, making high-frequency state updates a dominant source of latency, contention, and operational cost. In this work, we decouple inference from state persistence in streaming Machine Learning pipelines via probabilistic thinning: every event is scored, but durable state updates are selectively triggered by informative events. Unlike approaches that shed input or state, we show that persistence-path control is achievable without a high-frequency in-memory control plane or cross-worker coordination, relying exclusively on approximate statistics retrieved from disk-backed key-value stores. We model the resulting stochastic processes, derive bounds on filtering rates, and prove that common time-based aggregations remain unbiased under variance-aware formulations, preventing systemic error accumulation. We evaluate the approach in a controlled setting that isolates per-event costs, demonstrating substantial reductions in storage Input/Output and serialization overhead. Across experiments, up to 90% of events are excluded from the persistence path while preserving and in some cases improving downstream utility.

Problem

• low-latency feature engines
• state updates
• streaming machine learning
• persistence overhead
• probabilistic thinning

Innovation

• probabilistic thinning
• decoupled inference
• streaming machine learning
• state persistence
• low-latency feature engine