CommCP: Efficient Multi-Agent Coordination via LLM-Based Communication with Conformal Prediction

2026-02-05

AI Summary

This work addresses the inefficiency in multi-agent collaboration under natural language instructions, which often stems from redundant and unreliable communication. To tackle this issue, we propose CommCP—a novel framework that, for the first time, integrates conformal prediction into a large language model (LLM)-driven decentralized communication mechanism within the multi-agent multi-task embodied question answering (MM-EQA) setting. By calibrating message content to suppress irrelevant information, CommCP enhances communication precision and collaborative reliability. We introduce a new realistic home-scene benchmark tailored for MM-EQA and demonstrate that CommCP significantly outperforms existing baselines, achieving notable improvements in both task success rate and exploration efficiency.

Technology Category

• Multiagent Systems: Agent Communication
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Agentic search

Abstract

To complete assignments provided by humans in natural language, robots must interpret commands, generate and answer relevant questions for scene understanding, and manipulate target objects. Real-world deployments often require multiple heterogeneous robots with different manipulation capabilities to handle different assignments cooperatively. Beyond the need for specialized manipulation skills, effective information gathering is important in completing these assignments. To address this component of the problem, we formalize the information-gathering process in a fully cooperative setting as an underexplored multi-agent multi-task Embodied Question Answering (MM-EQA) problem, which is a novel extension of canonical Embodied Question Answering (EQA), where effective communication is crucial for coordinating efforts without redundancy. To address this problem, we propose CommCP, a novel LLM-based decentralized communication framework designed for MM-EQA. Our framework employs conformal prediction to calibrate the generated messages, thereby minimizing receiver distractions and enhancing communication reliability. To evaluate our framework, we introduce an MM-EQA benchmark featuring diverse, photo-realistic household scenarios with embodied questions. Experimental results demonstrate that CommCP significantly enhances the task success rate and exploration efficiency over baselines. The experiment videos, code, and dataset are available on our project website: https://comm-cp.github.io.

Problem

• multi-agent coordination
• Embodied Question Answering
• information gathering
• heterogeneous robots
• cooperative communication

Innovation

• Conformal Prediction
• LLM-based Communication
• Multi-Agent Coordination
• Embodied Question Answering
• Decentralized Framework