From Raw Data to Shared 3D Semantics: Task-Oriented Communication for Multi-Robot Collaboration

2026-02-09

AI Summary

This work addresses the challenge of communication congestion and high latency in multi-robot systems operating in complex 3D environments, which arise from the exchange of raw perceptual data and severely hinder collaborative efficiency. To overcome this limitation, the study introduces task-oriented semantic communication into multi-robot 3D collaboration for the first time, proposing a decentralized framework wherein each robot leverages a lightweight PiDiNet architecture combined with geometric processing to locally extract and share only task-relevant semantic information. This approach enables the construction of compact 3D scene representations that support coordinated perception, navigation, and object transportation. Experimental results demonstrate a dramatic reduction in communication overhead—from 858.6 Mb to 4.0 Mb (over 200× compression)—and a significant decrease in task completion steps from 1,054 to 281, thereby substantially enhancing collaborative performance.

Technology Category

• Intelligent Robots: Multi-Robot Systems
• Multiagent Systems: Agent Communication
• Computer Vision: 3D Computer Vision

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Multi-robot systems (MRS) rely on exchanging raw sensory data to cooperate in complex three-dimensional (3D) environments. However, this strategy often leads to severe communication congestion and high transmission latency, significantly degrading collaboration efficiency. This paper proposes a decentralized task-oriented semantic communication framework for multi-robot collaboration in unknown 3D environments. Each robot locally extracts compact, task-relevant semantics using a lightweight Pixel Difference Network (PiDiNet) with geometric processing. It shares only these semantic updates to build a task-sufficient 3D scene representation that supports cooperative perception, navigation, and object transport. Our numerical results show that the proposed method exhibits a dramatic reduction in communication overhead from $858.6$ Mb to $4.0$ Mb (over $200\times$ compression gain) while improving collaboration efficiency by shortening task completion from $1,054$ to $281$ steps.

Problem

• multi-robot systems
• communication congestion
• 3D environments
• task-oriented communication
• collaboration efficiency

Innovation

• task-oriented semantic communication
• multi-robot collaboration
• 3D scene representation
• lightweight semantic extraction
• communication-efficient MRS