A Predictive View on Streaming Hidden Markov Models

2026-04-10

AI Summary

This work addresses the challenge of online state inference and prediction in streaming hidden Markov models (HMMs) by proposing a prediction-oriented optimization framework. Under fixed state transition priors, the approach directly models predictive distributions through an online learning mechanism. The key contributions include establishing, for the first time, a theoretical foundation for beam search in HMMs based on forward KL-divergence optimal mixing, designing a fully recursive and deterministic streaming inference algorithm, and incorporating closed-form predictive updates that circumvent conventional EM or sampling procedures. Experimental results demonstrate that, under identical computational budgets, the proposed method outperforms online EM and sequential Monte Carlo approaches in sequence prediction accuracy.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

We develop a predictive-first optimisation framework for streaming hidden Markov models. Unlike classical approaches that prioritise full posterior recovery under a fully specified generative model, we assume access to regime-specific predictive models whose parameters are learned online while maintaining a fixed transition prior over regimes. Our objective is to sequentially identify latent regimes while maintaining accurate step-ahead predictive distributions. Because the number of possible regime paths grows exponentially, exact filtering is infeasible. We therefore formulate streaming inference as a constrained projection problem in predictive-distribution space: under a fixed hypothesis budget, we approximate the full posterior predictive by the forward-KL optimal mixture supported on $S$ paths. The solution is the renormalised top-$S$ posterior-weighted mixture, providing a principled derivation of beam search for HMMs. The resulting algorithm is fully recursive and deterministic, performing beam-style truncation with closed-form predictive updates and requiring neither EM nor sampling. Empirical comparisons against Online EM and Sequential Monte Carlo under matched computational budgets demonstrate competitive prequential performance.

Problem

• streaming hidden Markov models
• latent regime identification
• predictive distributions
• sequential inference
• exponential path growth

Innovation

• predictive inference
• streaming hidden Markov models
• beam search
• forward-KL projection
• online learning