OnEvoMemory: Evolving Memory through Online Robot Rollouts for Pretrained Robot Policies

2026-08-09

AI Summary

This work addresses the challenge of long-horizon robotic manipulation, where existing policies struggle to effectively track task progress and critical events, often relying on external models or handcrafted rules for memory updates. To overcome this limitation, the authors propose OnEvoMemory—a value-guided memory module that uniquely leverages online interaction outcomes to drive memory evolution. Initialized from offline demonstrations, OnEvoMemory dynamically refines its content by incorporating both successful and failed online trajectories. Operating without external models, it adaptively selects relevant information and seamlessly integrates into vision-language-action (VLA) policy frameworks. Empirical evaluation demonstrates substantial performance gains on long-horizon manipulation benchmarks, highlighting the synergistic benefits of offline initialization coupled with online memory evolution.

Abstract

Long-horizon robot manipulation requires policies to track completed subtasks and critical interaction events. However, existing memory mechanisms heavily rely on external models or predefined update rules. To address this, we propose OnEvoMemory, a value-guided memory module for pretrained robot policies. It maintains recent context, high-value experiences, and salient transitions, while learning which experiences should be retained from trajectory outcomes. Offline demonstrations initialize the memory prior, whereas successful and unsuccessful online rollouts refine memory selection, helping the policy recognize task-stage transitions and avoid repeating completed subtasks. Experiments on long-horizon manipulation benchmarks show that OnEvoMemory improves the performance of the base VLA policy through both offline initialization and online memory evolution.

Problem

• long-horizon manipulation
• memory mechanism
• robot policies
• task-stage transitions
• subtask tracking

Innovation

• OnEvoMemory
• online rollouts
• memory evolution
• pretrained robot policies
• long-horizon manipulation