Asymptotically Optimal Sequential Testing with Markovian Data

2026-02-19

AI Summary

This work addresses the problem of one-sided α-correct sequential hypothesis testing for data generated by ergodic Markov chains, where the null and alternative hypotheses correspond to disjoint sets of transition matrices. For the first time, it jointly incorporates the stationary distribution and transition structure into a non-asymptotic lower bound analysis, establishing a tight instance-dependent lower bound. Leveraging information-theoretic tools, ergodic theory of Markov chains, and sequential testing design, the paper proposes an optimal test whose expected stopping time asymptotically achieves this bound as α → 0. This approach overcomes the limitations of existing methods that are either only asymptotically optimal or suboptimal, providing a sharp characterization of sequential testing for Markovian data. The framework is successfully applied to detecting misspecification in MCMC models and sequentially verifying linear structural assumptions on transition dynamics in Markov decision processes.

Technology Category

• Reasoning under Uncertainty: Sequential Decision Making
• Machine Learning: Probabilistic Circuits and Graphical Models
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements

Abstract

We study one-sided and $α$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the alternative corresponds to a disjoint set $Q$. We establish a tight non-asymptotic instance-dependent lower bound on the expected stopping time of any valid sequential test under the alternative. Our novel analysis improves the existing lower bounds, which are either asymptotic or provably sub-optimal in this setting. Our lower bound incorporates both the stationary distribution and the transition structure induced by the unknown Markov chain. We further propose an optimal test whose expected stopping time matches this lower bound asymptotically as $α\to 0$. We illustrate the usefulness of our framework through applications to sequential detection of model misspecification in Markov Chain Monte Carlo and to testing structural properties, such as the linearity of transition dynamics, in Markov decision processes. Our findings yield a sharp and general characterization of optimal sequential testing procedures under Markovian dependence.

Problem

• sequential hypothesis testing
• Markovian data
• expected stopping time
• transition matrix
• ergodic Markov chain

Innovation

• sequential hypothesis testing
• Markovian data
• asymptotically optimal
• instance-dependent lower bound
• stopping time