GOMP: Grasped Object Manifold Projection for Multimodal Imitation Learning of Manipulation

2025-12-03

AI Summary

To address trajectory inaccuracy in imitation learning caused by error accumulation during precision assembly, this paper proposes a novel imitation learning framework constrained by a low-dimensional manifold of the manipulated object. The core innovation is the first introduction of a manifold projection mechanism that constrains object motion to a task-relevant low-dimensional manifold, coupled with an *n*-armed bandit algorithm for dynamic policy adaptation—effectively suppressing error propagation without requiring additional labeled data. The method integrates multimodal perception (including tactile sensing), non-rigid object modeling, and manifold-aware optimization, yielding strong generalization capability. Evaluated on four high-precision assembly tasks, the approach achieves significant improvements in both success rate and trajectory accuracy, demonstrating its cross-modal effectiveness and engineering practicality.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Imitation Learning (IL) holds great potential for learning repetitive manipulation tasks, such as those in industrial assembly. However, its effectiveness is often limited by insufficient trajectory precision due to compounding errors. In this paper, we introduce Grasped Object Manifold Projection (GOMP), an interactive method that mitigates these errors by constraining a non-rigidly grasped object to a lower-dimensional manifold. GOMP assumes a precise task in which a manipulator holds an object that may shift within the grasp in an observable manner and must be mated with a grounded part. Crucially, all GOMP enhancements are learned from the same expert dataset used to train the base IL policy, and are adjusted with an n-arm bandit-based interactive component. We propose a theoretical basis for GOMP's improvement upon the well-known compounding error bound in IL literature. We demonstrate the framework on four precise assembly tasks using tactile feedback, and note that the approach remains modality-agnostic. Data and videos are available at williamvdb.github.io/GOMPsite.

Problem

• Improves imitation learning precision for repetitive manipulation tasks.
• Reduces compounding errors via lower-dimensional manifold constraints.
• Enables precise object mating using multimodal feedback and expert data.

Innovation

• Constrains grasped object to lower-dimensional manifold
• Learns enhancements from same expert dataset
• Uses n-arm bandit-based interactive adjustment