Large Language Models over Networks: Collaborative Intelligence under Resource Constraints

2026-05-08

AI Summary

This work addresses the challenge that a single endpoint under heterogeneous resource constraints struggles to simultaneously satisfy diverse requirements such as intermittent connectivity, low latency, data locality, and high-throughput inference. To overcome this limitation, the paper proposes a collaborative intelligence paradigm that establishes composable vertical (device–cloud) and horizontal (multi-agent) collaboration dimensions, forming a large language model (LLM) cooperative inference framework tailored for resource-constrained networks. The framework enables efficient multi-model coordination through task-level communication—via natural language or structured messages—adaptive routing policy learning, and hybrid topology deployment. Experimental results demonstrate that the proposed approach significantly enhances response quality, offering a viable solution for deploying LLM services in scenarios characterized by low connectivity, high workload, and stringent latency sensitivity.

Abstract

Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving. Yet cloud-based LLM services alone cannot serve a growing class of applications, including those operating under intermittent connectivity, sub-second latency budgets, data-residency constraints, or sustained high-volume inference. On-device deployment is in turn constrained by limited computation and memory. No single endpoint can deliver high-quality service across this spectrum. This article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages. Such collaboration strives for superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers. We present collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration, which can be combined into hybrid topologies in practice. We then examine learning to collaborate, addressing the training of routing policies and the development of cooperative capabilities among LLMs. Finally, we identify open research challenges including scaling under resource heterogeneity and trustworthy collaborative intelligence.

Problem

• Large Language Models
• Collaborative Intelligence
• Resource Constraints
• On-device Deployment
• Networked AI

Innovation

• Collaborative Intelligence
• Large Language Models
• Resource-Constrained Inference
• Device-Cloud Collaboration
• Multi-Agent LLMs