Beyond Needle(s) in the Embodied Haystack: Environment, Architecture, and Training Considerations for Long Context Reasoning

2025-05-22

AI Summary

To address the weak long-context reasoning capability of embodied AI in hundred-step, long-horizon tasks, this paper proposes ∞-THOR. Methodologically, it introduces: (1) the first hundred-step embodied long-horizon reasoning benchmark, featuring a novel “Needle-in-an-Embodied-Haystack” multi-clue embodied question-answering task and establishing the first cross-trajectory, multi-clue reasoning evaluation paradigm; (2) a target-state-action interleaved modeling architecture with context parallelism, overcoming LLMs’ contextual bottlenecks in extended interactive trajectories; and (3) a scalable synthetic trajectory generation mechanism. Empirical results reveal a systematic performance degradation in existing models as step count increases, and demonstrate that ∞-THOR significantly outperforms baselines in multi-clue localization and long-term planning, validating the efficacy of its architectural innovations.

Technology Category

• Intelligent Robots: Embodied AI
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We introduce $infty$-THOR, a new framework for long-horizon embodied tasks that advances long-context understanding in embodied AI. $infty$-THOR provides: (1) a generation framework for synthesizing scalable, reproducible, and unlimited long-horizon trajectories; (2) a novel embodied QA task, Needle(s) in the Embodied Haystack, where multiple scattered clues across extended trajectories test agents' long-context reasoning ability; and (3) a long-horizon dataset and benchmark suite featuring complex tasks that span hundreds of environment steps, each paired with ground-truth action sequences. To enable this capability, we explore architectural adaptations, including interleaved Goal-State-Action modeling, context extension techniques, and Context Parallelism, to equip LLM-based agents for extreme long-context reasoning and interaction. Experimental results and analyses highlight the challenges posed by our benchmark and provide insights into training strategies and model behaviors under long-horizon conditions. Our work provides a foundation for the next generation of embodied AI systems capable of robust, long-term reasoning and planning.

Problem

• Develops a framework for long-horizon embodied AI tasks
• Tests agents' reasoning with scattered clues in extended trajectories
• Explores architectural adaptations for extreme long-context reasoning

Innovation

• Generates scalable long-horizon trajectories for training
• Introduces novel Needle(s) in Embodied Haystack QA task
• Uses interleaved Goal-State-Action modeling for long-context