WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agents

2026-06-17

AI Summary

Existing benchmarks struggle to evaluate embodied agents’ ability to retain long-term memory of user habits, world states, and interaction histories in dynamic home environments and leverage this memory for task execution. To address this gap, this work introduces WorldLines—the first embodied intelligence benchmark tailored for long-term household assistance—featuring temporally extended in-home trajectories and the ObsMem memory framework. ObsMem unifies long-term memory with embodied task planning by integrating visibility-aware memory with action-native state trajectories, constructing evidence-linked samples from dialogue, actions, feedback, and environmental state changes. Experimental results demonstrate that ObsMem significantly outperforms baseline methods in addressing core challenges such as partial observability, comprehensive world state coverage, and memory-driven planning.

Abstract

To assist humans over extended periods in real homes, embodied agents must remember user routines, world states, and past interactions. Existing long-term memory benchmarks mainly evaluate language-centric retrieval and question answering, while embodied benchmarks often focus on short-horizon task execution without testing long-term memory use in dynamic environments. We introduce WorldLines, a project-driven benchmark for long-horizon embodied household assistance. It constructs temporally extended household traces with dialogues, actions, execution feedback, object and device state changes, and converts them into evidence-linked samples for Memory QA and Embodied Task Planning. We further propose ObsMem, an observer-grounded memory framework that maintains visibility-aware memories and action-native state trails for state-aware decisions. Experiments reveal persistent challenges in partial observability, overwritten world states, and translating long-term memory into embodied plans, while ObsMem offers a stronger reference architecture for this setting.

Problem

• long-horizon
• stateful embodied agents
• memory benchmark
• dynamic environments
• household assistance

Innovation

• long-horizon embodied agents
• stateful memory
• WorldLines benchmark
• ObsMem framework
• embodied task planning