RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

2026-03-04

AI Summary

Current vision-language-action (VLA) models lack standardized evaluation of memory capabilities, hindering systematic progress in long-horizon, history-dependent robotic manipulation tasks. This work proposes RoboMME—the first large-scale, standardized benchmark for assessing general-purpose policy memory—encompassing 16 tasks spanning temporal, spatial, object, and procedural memory dimensions. The authors also develop 14 memory-augmented VLA variants based on π0.5, exploring modular memory representations and diverse integration strategies. Experimental results reveal that different memory mechanisms exhibit task-specific performance advantages, with no universally optimal solution, thereby underscoring the critical importance of task-aware memory design. RoboMME thus establishes a new foundation and provides actionable directions for future research in robotic memory systems.

Abstract

Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipulating objects that become temporarily occluded. Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms; however, their evaluations remain confined to narrow, non-standardized settings. This limits their systematic understanding, comparison, and progress measurement. To address these challenges, we introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models in long-horizon, history-dependent scenarios. Our benchmark comprises 16 manipulation tasks constructed under a carefully designed taxonomy that evaluates temporal, spatial, object, and procedural memory. We further develop a suite of 14 memory-augmented VLA variants built on the {\pi}0.5 backbone to systematically explore different memory representations across multiple integration strategies. Experimental results show that the effectiveness of memory representations is highly task-dependent, with each design offering distinct advantages and limitations across different tasks. Videos and code can be found at our website https://robomme.github.io.

Problem

• memory
• robotic manipulation
• vision-language-action models
• long-horizon tasks
• benchmarking

Innovation

• memory-augmented VLA
• standardized benchmark
• long-horizon manipulation
• history-dependent tasks
• memory representation