Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decomposition

2026-04-09

AI Summary

Existing uncertainty quantification methods struggle to address the reliability challenges posed by multi-step reasoning, dynamic communication pathways, and diverse topological structures in large language model–based multi-agent systems. This work proposes MATU, a novel framework that, for the first time, introduces higher-order tensor decomposition to this domain. By modeling complete reasoning trajectories across multiple runs, MATU organizes agent embeddings into a tensor structure to disentangle and jointly quantify uncertainties arising from reasoning, communication, and topology. Experimental results demonstrate that MATU consistently yields comprehensive and robust uncertainty estimates across a variety of tasks and communication topologies, significantly outperforming existing approaches that focus solely on single-round outputs.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Multiagent Systems: Multiagent Systems under Uncertainty
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

While Large Language Model-based Multi-Agent Systems (MAS) consistently outperform single-agent systems on complex tasks, their intricate interactions introduce critical reliability challenges arising from communication dynamics and role dependencies. Existing Uncertainty Quantification methods, typically designed for single-turn outputs, fail to address the unique complexities of the MAS. Specifically, these methods struggle with three distinct challenges: the cascading uncertainty in multi-step reasoning, the variability of inter-agent communication paths, and the diversity of communication topologies. To bridge this gap, we introduce MATU, a novel framework that quantifies uncertainty through tensor decomposition. MATU moves beyond analyzing final text outputs by representing entire reasoning trajectories as embedding matrices and organizing multiple execution runs into a higher-order tensor. By applying tensor decomposition, we disentangle and quantify distinct sources of uncertainty, offering a comprehensive reliability measure that is generalizable across different agent structures. We provide comprehensive experiments to show that MATU effectively estimates holistic and robust uncertainty across diverse tasks and communication topologies.

Problem

• Uncertainty Quantification
• Multi-Agent Systems
• Large Language Models
• Communication Topology
• Cascading Uncertainty

Innovation

• tensor decomposition
• uncertainty quantification
• multi-agent systems
• LLM-based reasoning
• communication topology