OpenSPM: An Environment-Transferable Robotic Key Spatial Pose Memory and Closed-Loop High-Frequency Flow-Matching Action Generation Model

2026-06-29

AI Summary

This work addresses the challenge of robotic manipulation in open-world tabletop settings, which demands integrated semantic understanding, accurate 6D pose estimation, and high-frequency action generation. Existing end-to-end vision-language-action models often lack explicit geometric constraints and incur prohibitive training costs. To overcome these limitations, the authors propose a Spatial Persistent Memory framework that leverages semantic-conditioned 3D perception and Kalman filtering for robust pose tracking. The method extracts transferable key-pose memories from human demonstrations, retrieves them based on language instructions, and applies SE(3) transformations. High-frequency actions are generated via a lightweight flow-matching model, while proprioceptive feedback and end-effector residual correction enable closed-loop control. Evaluated on ten LIBERO-GOAL tasks, the approach achieves an 85.6% success rate with an effective control frequency of 1033.3 Hz, demonstrating strong environmental transferability, real-time performance, and low computational overhead.

Abstract

Open-environment tabletop robotic manipulation requires systems to possess semantic understanding, precise geometric pose estimation, and high-frequency action generation. While end-to-end vision-language-action (VLA) models excel at semantic generalization, they often lack explicit geometric constraints for fine-grained tasks and require costly training. To bridge the gap between high-level semantics and low-level physical execution, we propose OpenSPM, an open environment spatial persistent memory framework consisting of spatial pose memory and flow-matching action generation model. OpenSPM first leverages semantically conditioned 3D perception and Kalman filtering to track continuous 6D poses. It then extracts key spatial poses from human demonstrations, keeping them as transferable, object-centric spatial persistent memory entries. During inference, OpenSPM retrieves relevant memory entries in terms of natural language instructions, transfers the spatial poses to new scenes using SE(3) transformations, and generates high-frequency action chunks via a lightweight conditional flow-matching model. Combined with real-time proprioceptive state feedback and terminal residual correction, the system effectively suppresses trajectory error accumulation. Evaluated on ten LIBERO-GOAL tasks, OpenSPM achieves an 85.6% success rate and an equivalent control frequency of 1033.3 Hz, while requiring minimal inference AI computing power. Extensive ablations illustrate that structured spatial persistent memory and closed-loop residual correction play a crucial role in reliable, high-frequency robotic manipulation.

Problem

• robotic manipulation
• spatial pose estimation
• high-frequency action generation
• semantic understanding
• geometric constraints

Innovation

• spatial pose memory
• flow-matching
• closed-loop control
• SE(3) transformation
• high-frequency action generation