Decentralized Hidden Markov Modeling with Equal Exit Probabilities

2025-03-15

AI Summary

This paper addresses distributed social learning under dynamic environments, where agents must collectively infer a time-varying hidden state that evolves according to an equally-likely-exit Markov chain, using only private observations and local belief exchanges with neighbors. To this end, we propose Diffusion α-HMM—a novel distributed algorithm that introduces the equally-likely-exit constraint into decentralized HMM modeling for the first time. By parameterizing the nonlinear dynamics of log-belief ratios, it establishes a linearized connection to adaptive social learning. We prove that the algorithm achieves asymptotic consensus under time-varying network topologies, and establish existence and uniqueness of its fixed point. Numerical experiments demonstrate that Diffusion α-HMM significantly improves belief aggregation accuracy and robustness in tracking time-varying states across diverse dynamic scenarios, outperforming existing distributed HMM and social learning methods.

Technology Category

• Multiagent Systems: Multiagent Learning
• Intelligent Robots: State Estimation
• Machine Learning: Distributed Machine Learning & Federated Learning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

Social learning strategies enable agents to infer the underlying true state of nature in a distributed manner by receiving private environmental signals and exchanging beliefs with their neighbors. Previous studies have extensively focused on static environments, where the underlying true state remains unchanged over time. In this paper, we consider a dynamic setting where the true state evolves according to a Markov chain with equal exit probabilities. Based on this assumption, we present a social learning strategy for dynamic environments, termed Diffusion $alpha$-HMM. By leveraging a simplified parameterization, we derive a nonlinear dynamical system that governs the evolution of the log-belief ratio over time. This formulation further reveals the relationship between the linearized form of Diffusion $alpha$-HMM and Adaptive Social Learning, a well-established social learning strategy for dynamic environments. Furthermore, we analyze the convergence and fixed-point properties of a reference system, providing theoretical guarantees on the learning performance of the proposed algorithm in dynamic settings. Numerical experiments compare various distributed social learning strategies across different dynamic environments, demonstrating the impact of nonlinearity and parameterization on learning performance in a range of dynamic scenarios.

Problem

• Develops social learning for dynamic environments
• Analyzes convergence of learning algorithms
• Compares learning strategies in dynamic scenarios

Innovation

• Decentralized Hidden Markov Modeling
• Dynamic social learning strategy
• Nonlinear dynamical system analysis