Distributed Thompson sampling under constrained communication

2024-10-21

IEEE Control Systems Letters

AI Summary

This paper addresses Bayesian optimization in multi-agent systems under communication constraints. Method: We propose a distributed Thompson sampling framework wherein each agent maintains a local Gaussian process model and exchanges only sampled points over a graph-structured network. Contribution/Results: Theoretically, we establish the first communication-graph-dependent Bayesian regret bound, proving that—under graph connectivity—the distributed protocol achieves faster time convergence than single-agent Bayesian optimization, thereby overcoming limitations of conventional batch settings; our analysis implicitly incorporates graph signal processing principles to characterize information propagation. Empirically, experiments on standard benchmark functions demonstrate that graph connectivity significantly enhances collaborative optimization efficiency and convergence speed. The core innovation lies in explicitly embedding communication topology into the distributed sampling mechanism and providing the first regret analysis framework aware of graph structure.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Multiagent Systems: Agent Communication
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In Bayesian optimization, a black-box function is maximized via the use of a surrogate model. We apply distributed Thompson sampling, using a Gaussian process as a surrogate model, to approach the multi-agent Bayesian optimization problem. In our distributed Thompson sampling implementation, each agent receives sampled points from neighbors, where the communication network is encoded in a graph; each agent utilizes their own Gaussian process to model the objective function. We demonstrate theoretical bounds on Bayesian average regret and Bayesian simple regret, where the bound depends on the structure of the communication graph. Unlike in batch Bayesian optimization, this bound is applicable in cases where the communication graph amongst agents is constrained. When compared to sequential single-agent Thompson sampling, our bound guarantees faster convergence with respect to time as long as the communication graph is connected. We confirm the efficacy of our algorithm with numerical simulations on traditional optimization test functions, demonstrating the significance of graph connectivity on improving regret convergence.

Problem

• Distributed Thompson Sampling
• Bayesian Optimization
• Collaborative Search

Innovation

• Distributed Thompson Sampling
• Gaussian Processes
• Communication Constraints