StableIDM: Stabilizing Inverse Dynamics Model against Manipulator Truncation via Spatio-Temporal Refinement

2026-04-20

AI Summary

This work addresses the instability and ill-posed state recovery in inverse dynamics models for robotic arms under partial observability caused by visual occlusion. To stabilize action prediction in such truncated visual states, the authors propose a spatiotemporal refinement framework that integrates three core components: robot-centric masking, directional feature aggregation (DFA), and temporal dynamic refinement (TDR). By incorporating directional spatial reasoning and temporal continuity constraints, the framework enhances the robustness of vision-to-action mapping. Experimental results on the AgiBot benchmark demonstrate significant improvements: a 12.1% increase in strict action accuracy, a 9.7% gain in real-world task success rate, an 11.5% improvement in end-to-end grasping success, and a 17.6% boost in downstream vision-language-action (VLA) task performance.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Inverse Dynamics Models (IDMs) map visual observations to low-level action commands, serving as central components for data labeling and policy execution in embodied AI. However, their performance degrades severely under manipulator truncation, a common failure mode that makes state recovery ill-posed and leads to unstable control. We present StableIDM, a spatio-temporal framework that refines features from visual inputs to stabilize action predictions under such partial observability. StableIDM integrates three complementary components: (1) auxiliary robot-centric masking to suppress background clutter, (2) Directional Feature Aggregation (DFA) for geometry-aware spatial reasoning, which extracts anisotropic features along directions inferred from the visible arm and (3) Temporal Dynamics Refinement (TDR) to smooth and correct predictions via motion continuity. Extensive evaluations validate our approach: StableIDM improves strict action accuracy by 12.1% under severe truncation on the AgiBot benchmark, and increases average task success by 9.7% in real-robot replay. Moreover, it boosts end-to-end grasp success by 11.5% when decoding video-generated plans, and improves downstream VLA real-robot success by 17.6% when functioning as an automatic annotator. These results demonstrate that StableIDM provides a robust and scalable backbone for both policy execution and data generation in embodied artificial intelligence.

Problem

• Inverse Dynamics Model
• Manipulator Truncation
• Partial Observability
• Embodied AI
• Control Stability

Innovation

• Inverse Dynamics Model
• Manipulator Truncation
• Spatio-Temporal Refinement
• Directional Feature Aggregation
• Temporal Dynamics Refinement