Learning To Communicate Over An Unknown Shared Network

2025-07-08

AI Summary

Multi-agent communication scheduling in unknown shared wireless networks (e.g., Wi-Fi or cellular) is challenging due to partial observability, sparse feedback (e.g., ACKs, RTT), and dynamic agent population size. Method: We propose Query Net (QNet), a generalizable deep reinforcement learning policy that relies solely on per-agent local observations and employs minimal parameterization combined with a simulation-to-real transfer training framework. QNet adaptively schedules transmissions for arbitrary numbers of concurrent agents without retraining for specific network configurations or agent counts. Results: Evaluated in Wi-Fi scenarios with 5–50 agents and cellular networks with RTT ranging from 0.07 to 0.83 seconds, QNet consistently outperforms state-of-the-art baselines across low-to-high channel contention regimes. Its performance demonstrates strong generalization across varying network conditions, agent scales, and wireless technologies—validating both practical applicability and robustness in real-world deployments.

Technology Category

• Multiagent Systems: Agent Communication
• Planning, Routing, and Scheduling: Planning/Scheduling and Learning
• Search and Optimization: Learning to Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments

Abstract

As robots (edge-devices, agents) find uses in an increasing number of settings and edge-cloud resources become pervasive, wireless networks will often be shared by flows of data traffic that result from communication between agents and corresponding edge-cloud. In such settings, agent communicating with the edge-cloud is unaware of state of network resource, which evolves in response to not just agent's own communication at any given time but also to communication by other agents, which stays unknown to the agent. We address challenge of an agent learning a policy that allows it to decide whether or not to communicate with its cloud node, using limited feedback it obtains from its own attempts to communicate, to optimize its utility. The policy generalizes well to any number of other agents sharing the network and must not be trained for any particular network configuration. Our proposed policy is a DRL model Query Net (QNet) that we train using a proposed simulation-to-real framework. Our simulation model has just one parameter and is agnostic to specific configurations of any wireless network. It allows training an agent's policy over a wide range of outcomes that an agent's communication with its edge-cloud node may face when using a shared network, by suitably randomizing the simulation parameter. We propose a learning algorithm that addresses challenges observed in training QNet. We validate our simulation-to-real driven approach through experiments conducted on real wireless networks including WiFi and cellular. We compare QNet with other policies to demonstrate its efficacy. WiFi experiments involved as few as five agents, resulting in barely any contention for the network, to as many as fifty agents, resulting in severe contention. The cellular experiments spanned a broad range of network conditions, with baseline RTT ranging from a low of 0.07 second to a high of 0.83 second.

Problem

• Agents learn to communicate over unknown shared networks.
• Policy generalizes to any number of agents sharing network.
• DRL model QNet optimizes utility with limited feedback.

Innovation

• DRL model QNet for communication policy
• Simulation-to-real framework with one parameter
• Generalizes to any shared network configuration