Object-Centric Environment Modeling for Agentic Tasks

2026-07-02

AI Summary

This work addresses the challenge that large language model agents struggle to effectively maintain, verify, and reuse free-form textual memories over long-term interactions, while existing symbolic approaches are often confined to localized procedures or simplified environments. The authors propose an object-centric memory modeling paradigm that structures experience into executable object knowledge—represented as Python classes encoding environmental entities—and procedural knowledge capturing reusable interaction patterns. These two knowledge forms co-evolve and are updated and validated online after each task. Through mechanisms including programmatic skill logging, trajectory reflection, code-level verification, and on-demand source code inspection, the system enables progressive knowledge disclosure. Experiments demonstrate that this approach achieves state-of-the-art average performance across multiple benchmarks, significantly reduces ineffective actions, and validates the benefits of object-centric modeling for enhancing the scalability and efficacy of agent memory systems.

Abstract

Large language model (LLM) agents can improve through accumulated experience, but free-form textual memories become difficult to maintain, validate, and reuse as interactions grow. Recent symbolic approaches learn executable skills or programmatic world models, yet often store local procedures or assume simplified dynamics. We propose Object-Centric Environment Modeling (OCM), which organizes experience into an executable object-centric environment model. OCM maintains two connected code bases: object knowledge, which defines environment entities and mechanisms as Python classes, and procedure knowledge, which records reusable interaction patterns that must import and use the object model. OCM works in an online setting: after each episode, OCM reflects on the trajectory, updates both knowledge bases, and verifies that all procedures execute against the updated object model. During future interaction, the agent uses progressive knowledge disclosure to inspect compact code signatures first and read source code only when needed. Experiments show that OCM achieves the best average rank across benchmarks and reduces invalid actions, demonstrating that agents can benefit from building object-centric environment models.

Problem

• object-centric modeling
• environment modeling
• LLM agents
• memory organization
• executable world models

Innovation

• Object-Centric Modeling
• Executable World Models
• Online Knowledge Integration
• LLM Agents
• Progressive Knowledge Disclosure