TransAM: Transformer-Based Agent Modeling for Multi-Agent Systems via Local Trajectory Encoding

2025-08-04

AI Summary

To address the challenge of policy modeling in multi-agent systems due to the absence of global trajectory information, this paper proposes a novel method that infers other agents’ policies solely from the controlled agent’s local trajectories. Our core contribution is the first Transformer-based encoding framework tailored for local observations, which leverages self-supervised sequential modeling to robustly learn implicit policy embeddings of other agents from limited local trajectory data. The approach operates without access to global state or inter-agent communication, and is universally applicable to cooperative, competitive, and mixed-interaction settings. Experiments demonstrate that the learned policy representations significantly improve modeling accuracy (average +12.7%) and enhance long-horizon cumulative returns across diverse tasks (up to +18.3%). This work establishes a scalable new paradigm for multi-agent coordination and reasoning under resource constraints.

Technology Category

• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Agent modeling is a critical component in developing effective policies within multi-agent systems, as it enables agents to form beliefs about the behaviors, intentions, and competencies of others. Many existing approaches assume access to other agents' episodic trajectories, a condition often unrealistic in real-world applications. Consequently, a practical agent modeling approach must learn a robust representation of the policies of the other agents based only on the local trajectory of the controlled agent. In this paper, we propose exttt{TransAM}, a novel transformer-based agent modeling approach to encode local trajectories into an embedding space that effectively captures the policies of other agents. We evaluate the performance of the proposed method in cooperative, competitive, and mixed multi-agent environments. Extensive experimental results demonstrate that our approach generates strong policy representations, improves agent modeling, and leads to higher episodic returns.

Problem

• Agent modeling in multi-agent systems without full trajectory access
• Learning robust policy representations from local trajectories only
• Improving agent performance in cooperative competitive mixed environments

Innovation

• Transformer-based agent modeling approach
• Local trajectory encoding for policy representation
• Effective in cooperative and competitive environments