Analytic Concept-Centric Memory for Agentic Embodied Manipulation

2026-06-29

AI Summary

This work addresses the challenge faced by embodied agents in long-horizon tasks: effectively memorizing and retrieving information about object parts, states, action effects, and skills. To this end, the authors propose a structured memory framework centered on analytical concepts, organizing experiences into semantic parts, parameterized templates, embodied poses, functional affordances, and operational states. This unified representation captures scenes, state transitions, and skill memories within a single architecture, enabling coarse-to-fine retrieval. By introducing structured analytical concepts into embodied memory for the first time, the approach overcomes limitations of conventional unstructured or embedding-based methods. Empirical results demonstrate significant improvements over existing baselines in task completion rate, retrieval accuracy, object re-identification, and cross-object skill generalization.

Abstract

Long-horizon embodied manipulation requires agents to remember persistent objects, track changing scene states, and reuse prior interaction knowledge. However, existing agent memories are often stored as unstructured histories or embedding-based records, making it difficult to retrieve manipulation-relevant object parts, physical states, action effects, and executable skills. We propose an analytic concept-centric memory framework for agentic embodied manipulation. Our memory organizes experience around structured analytic concepts, where objects are represented by semantic parts, parametric templates, grounded poses, affordances, and manipulation states. It further connects object and scene memories with transition memory for action-induced state changes and skill memory for template-grounded and policy-grounded execution. At runtime, the agent performs structured coarse-to-fine retrieval to identify relevant objects, states, transitions, and skills, supporting state-consistent reasoning and skill reuse. Experiments on memory-dependent manipulation, articulated-object generalization, real-world memory evaluation, and ablations show that our approach improves task completion, retrieval accuracy, object re-identification, and cross-object skill generalization over unstructured and embedding-based memory baselines.

Problem

• embodied manipulation
• agent memory
• long-horizon tasks
• structured memory
• skill reuse

Innovation

• concept-centric memory
• structured retrieval
• embodied manipulation
• skill generalization
• analytic object representation