One-Shot Manipulation Strategy Learning by Making Contact Analogies

2024-11-14

arXiv.org

AI Summary

This paper addresses the challenge of cross-object generalization for dexterous manipulation policies under one-shot learning. We propose a contact-based analogical action transfer method: given a single demonstration trajectory, it automatically identifies physically plausible and functionally equivalent contact points and action sequences on novel objects, enabling generalization across diverse tasks such as hooking, hanging, and scooping. Our core contribution is a two-stage contact point matching mechanism that jointly leverages pre-trained shape embeddings and local geometric curvature analysis, ensuring robust matching across objects with varying deformations, scales, and categories. The method requires no fine-tuning and supports real-time inference. It significantly outperforms existing state-of-the-art approaches on three distinct manipulation tasks, generalizes to unseen object categories, and achieves substantial speedup in runtime performance.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Learning with Manifolds
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a novel approach, MAGIC (manipulation analogies for generalizable intelligent contacts), for one-shot learning of manipulation strategies with fast and extensive generalization to novel objects. By leveraging a reference action trajectory, MAGIC effectively identifies similar contact points and sequences of actions on novel objects to replicate a demonstrated strategy, such as using different hooks to retrieve distant objects of different shapes and sizes. Our method is based on a two-stage contact-point matching process that combines global shape matching using pretrained neural features with local curvature analysis to ensure precise and physically plausible contact points. We experiment with three tasks including scooping, hanging, and hooking objects. MAGIC demonstrates superior performance over existing methods, achieving significant improvements in runtime speed and generalization to different object categories. Website: https://magic-2024.github.io/ .

Problem

• One-shot learning of manipulation strategies for novel objects
• Identifying contact points and action sequences from reference trajectories
• Ensuring precise and physically plausible contact-point matching

Innovation

• One-shot learning with contact analogies
• Two-stage contact-point matching process
• Combines global shape and local curvature