Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraft

2026-06-14

AI Summary

This work addresses the challenge of collaborative failure among multi-agent systems in dynamic, heterogeneous environments with strict real-time constraints, primarily caused by partial observability. To this end, we introduce TickingCollabBench—the first unified benchmark that integrates heterogeneity, mandatory collaboration, environmental dynamics, and real-time constraints. Built upon Minecraft, the benchmark leverages YAML-based declarative task specifications, LLM-assisted task drafting, and a constraint-driven feasibility validator to enable automated generation of diverse collaborative tasks. Experimental results demonstrate that current LLM-based approaches significantly underperform a global oracle in dynamic real-time settings due to coordination difficulties and latency, thereby revealing critical limitations of existing methods and establishing a new paradigm for realistic multi-agent collaboration research.

Abstract

We present TickingCollabBench, a Minecraft-based multi-agent benchmark for a novel class of time-sensitive complementary collaboration tasks. Our benchmark reflects four core characteristics of real-world collaboration: agent heterogeneity, mandatory collaboration, dynamic environments, and strict real-time constraints with failure risks. To enable this, we develop the TickingCollab framework, which supports the generation of diverse dynamic environments and abstracts Minecraft's primitive APIs to enable declarative YAML task specifications for composing these events. Building on this, we design a feasibility-aware automated benchmark generation pipeline, where an LLM drafts structurally diverse task configurations and feasibility verifier filters out invalid ones using approximate constraints. Evaluations demonstrate that lang latency and inherent difficulty of coordinating under partial observability and agent heterogeneity cause LLMs to frequently fail under dynamic environments and fall significantly short of a global-knowledge oracle.

Problem

• multi-agent collaboration
• time-sensitive tasks
• agent heterogeneity
• dynamic environments
• partial observability

Innovation

• multi-agent collaboration
• time-sensitive tasks
• dynamic environments
• declarative task specification
• feasibility-aware generation