Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks

2026-01-19

arXiv.org

AI Summary

This work addresses the challenge of real-time sampling and autoregressive Markov source estimation in dynamic yet structurally similar multi-hop wireless networks, where optimal policy design is hindered by high-dimensional action spaces and complex network topologies. The authors propose a graph neural network–based multi-agent reinforcement learning framework that leverages a centralized training with decentralized execution paradigm and incorporates recurrent mechanisms to enable decentralized policy optimization aimed at minimizing time-averaged estimation error. The approach facilitates policy transfer across networks of varying scales but similar structures, exhibits robustness in non-stationary environments, and demonstrates performance improvements as the number of agents increases, significantly outperforming existing baselines.

Technology Category

• Multiagent Systems: Multiagent Learning
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Machine Learning: Reinforcement Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We address real-time sampling and estimation of autoregressive Markovian sources in dynamic yet structurally similar multi-hop wireless networks. Each node caches samples from others and communicates over wireless collision channels, aiming to minimize time-average estimation error via decentralized policies. Due to the high dimensionality of action spaces and complexity of network topologies, deriving optimal policies analytically is intractable. To address this, we propose a graphical multi-agent reinforcement learning framework for policy optimization. Theoretically, we demonstrate that our proposed policies are transferable, allowing a policy trained on one graph to be effectively applied to structurally similar graphs. Numerical experiments demonstrate that (i) our proposed policy outperforms state-of-the-art baselines; (ii) the trained policies are transferable to larger networks, with performance gains increasing with the number of agents; (iii) the graphical training procedure withstands non-stationarity, even when using independent learning techniques; and (iv) recurrence is pivotal in both independent learning and centralized training and decentralized execution, and improves the resilience to non-stationarity.

Problem

• decentralized learning
• estimation error minimization
• graph neural networks
• multi-hop wireless networks
• autoregressive Markovian sources

Innovation

• Graph Neural Networks
• Decentralized Reinforcement Learning
• Policy Transferability
• Non-stationary Environments
• Multi-agent Systems