ADKO: Agentic Decentralized Knowledge Optimization

2026-05-08

AI Summary

This work addresses key challenges in multi-agent black-box optimization—namely low sample efficiency, privacy leakage, difficulty in coordinating heterogeneous objectives, and high communication overhead—by proposing a decentralized collaborative optimization framework. Agents exchange compressed “knowledge tokens,” which encode directional signals and advantage scores, and jointly explore the solution space using Gaussian processes and language models, without sharing raw data or model parameters. The paper presents the first formal analysis of the dual information loss induced by token compression and language model approximation, establishes a regret decomposition theory, and introduces a fidelity-aware token pruning strategy. Experiments demonstrate that the proposed method significantly outperforms strong baselines on neural architecture search and scientific discovery tasks, achieving efficient, low-communication collaboration while preserving privacy.

Abstract

We present Agentic Decentralized Knowledge Optimization (ADKO), a framework for collaborative black-box optimization across autonomous agents that achieves sample efficiency, privacy preservation, heterogeneous-objective handling, and communication efficiency. Each agent maintains a private Gaussian Process (GP) surrogate trained on local data and communicates only through knowledge tokens-compact, lossy summaries containing directional signals, advantage scores, and optional language-model (LM) insights-without sharing raw data or model parameters. ADKO unifies GP-Upper Confidence Bound (GP-UCB), parallel Bayesian optimization, decentralized learning, and LM-guided discovery. We provide the first formal analysis of dual information loss: token compression, quantified via mutual-information-based fidelity, and LM approximation error, decomposed into bias and stochastic noise. Our main result shows cumulative regret decomposes into GP error, LM bias, LM noise, and compression loss, with necessary and sufficient conditions for sublinear regret. We also propose fidelity-aware token pruning to preserve high-information tokens under memory budget. Experiments on neural architecture search and scientific discovery validate the theory and show consistent improvements over strong baselines.

Problem

• black-box optimization
• decentralized learning
• privacy preservation
• heterogeneous objectives
• communication efficiency

Innovation

• Agentic Decentralized Knowledge Optimization
• Knowledge Tokens
• Gaussian Process Surrogate
• Mutual Information Fidelity
• Sublinear Regret