Enhancing Multi-Agent Communication through Attention Steering with Context Relevance

2026-05-28

AI Summary

This work addresses the performance degradation of multi-agent systems in long-horizon interactions, where lengthy dialogue histories dilute critical information with irrelevant context. To mitigate this issue, the authors propose Agent-Radar, a training-free context management method that dynamically steers agent attention toward highly relevant historical segments through a novel spatiotemporal decay mechanism. Built upon large language models, Agent-Radar integrates contextual relevance modeling with spatiotemporal decay strategies to prioritize informative past interactions. Evaluated across five benchmark tasks, the method consistently outperforms state-of-the-art approaches, achieving performance gains of up to 7.64 percentage points. Notably, it maintains both efficiency and robustness even as the number of agents and interaction rounds increases.

Abstract

LLM-based multi-agent systems have demonstrated remarkable performance on complex tasks through collaborative reasoning. However, these systems tend to rapidly accumulate extremely long conversation histories during interaction. As conversations lengthen, relevant information is increasingly diluted by irrelevant context, leading to degraded performance. In this work, we present Agent-Radar, a training-free context management method that dynamically steers each agent's attention toward relevant context with a novel temporal and spatial decay mechanism. Our experiments demonstrate that Agent-Radar outperforms state-of-the-art methods across five different benchmarks, yielding gains of up to 7.64 absolute points. Furthermore, our analysis shows that Agent-Radar remains effective and robust as the number of agents and interaction rounds increases. Finally, the ablation study shows that core components in Agent-Radar are crucial to performance and generalizable in different settings.

Problem

• multi-agent communication
• context relevance
• conversation history
• attention dilution
• LLM-based systems

Innovation

• Attention Steering
• Context Relevance
• Multi-Agent Communication
• Temporal Decay
• Spatial Decay