ECHO: Continuous Hierarchical Memory for Vision-Language-Action Models

2026-05-09

AI Summary

Existing vision-language-action models struggle with long-horizon manipulation tasks due to the absence of a structured memory mechanism, limiting their ability to effectively retrieve past experiences and generalize to unseen task compositions. This work proposes ECHO, a novel framework that introduces, for the first time, a continuous hierarchical memory space grounded in hyperbolic geometry. By employing a hyperbolic autoencoder, ECHO maps latent states into a semantic memory tree and dynamically refines its structure through mechanisms such as context-dependent consolidation, geometric interpolation, and structural splitting. This approach enables efficient memory retrieval and synthesis of counterfactual experiences, achieving an absolute success rate improvement of 12.8% over the π₀ baseline on LIBERO-Long and demonstrating substantially enhanced compositional generalization on unseen long-horizon tasks across suites.

Abstract

Memory capacity is a critical factor determining the performance of Vision-Language-Action (VLA) models in long-horizon manipulation tasks. Existing memory-augmented architectures primarily rely on linear or flat storage, lacking structural priors for manipulation categories and hierarchical organization. This deficiency hinders efficient experience retrieval and limits generalization to unseen long-horizon task compositions. Inspired by the hierarchical organization of human experience, we propose ECHO (Experience Consolidation and Hierarchical Organization), a novel memory framework operating within a Continuous Hierarchical Space. By employing a hyperbolic autoencoder, ECHO maps VLA hidden states into this space. Leveraging hyperbolic metrics and entailment constraint mechanisms, experience vectors are organized into a semantic memory tree that supports efficient top-down retrieval. In parallel, a background consolidation mechanism continuously refines the memory tree through geometric interpolation and structural splitting, supporting virtual memory synthesis in the continuous space. We integrate ECHO into the $π_0$ foundation model. Evaluations on LIBERO and preliminary real-world experiments demonstrate the effectiveness of our approach, notably achieving a 12.8% absolute improvement in execution success rate over the $π_0$ baseline on LIBERO-Long, while improving compositional generalization on cross-suite unseen long-horizon tasks.

Problem

• Vision-Language-Action models
• memory architecture
• hierarchical organization
• long-horizon tasks
• compositional generalization

Innovation

• Continuous Hierarchical Memory
• Hyperbolic Autoencoder
• Semantic Memory Tree
• Experience Consolidation
• Compositional Generalization