Gated Memory Policy

2026-04-20

AI Summary

This work addresses the distribution shift and overfitting issues that arise from naively extending observation histories in non-Markovian robotic manipulation tasks. To this end, the authors propose a memory-gated visuomotor policy that dynamically determines when to access historical information through a learnable gating mechanism. The approach integrates a lightweight cross-attention module to efficiently construct contextual representations and incorporates diffusion-based action noise to enhance robustness against historical errors. Evaluated on the MemMimic non-Markovian benchmark, the method achieves an average success rate improvement of 30.1% over long-history baselines, while maintaining strong performance on Markovian RoboMimic tasks, thereby demonstrating its generality and effectiveness.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Diffusion Models for Vision

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Robotic manipulation tasks exhibit varying memory requirements, ranging from Markovian tasks that require no memory to non-Markovian tasks that depend on historical information spanning single or multiple interaction trials. Surprisingly, simply extending observation histories of a visuomotor policy often leads to a significant performance drop due to distribution shift and overfitting. To address these issues, we propose Gated Memory Policy (GMP), a visuomotor policy that learns both when to recall memory and what to recall. To learn when to recall memory, GMP employs a learned memory gate mechanism that selectively activates history context only when necessary, improving robustness and reactivity. To learn what to recall efficiently, GMP introduces a lightweight cross-attention module that constructs effective latent memory representations. To further enhance robustness, GMP injects diffusion noise into historical actions, mitigating sensitivity to noisy or inaccurate histories during both training and inference. On our proposed non-Markovian benchmark MemMimic, GMP achieves a 30.1% average success rate improvement over long-history baselines, while maintaining competitive performance on Markovian tasks in RoboMimic. All code, data and in-the-wild deployment instructions are available on our project website https://gated-memory-policy.github.io/.

Problem

• robotic manipulation
• memory requirements
• non-Markovian tasks
• distribution shift
• overfitting

Innovation

• Gated Memory Policy
• memory gating
• cross-attention
• diffusion noise
• non-Markovian manipulation