Leveraging Neural Descriptor Fields for Learning Contact-Aware Dynamic Recovery

2025-10-16

AI Summary

Real-world dexterous manipulation is highly susceptible to unexpected disturbances, often causing object drops; thus, rapid interception and repositioning of falling objects—while still within the manipulator’s reachable workspace—is critical for task recovery. This paper proposes Contact-Aware Dynamic Recovery (CADRE), a novel framework that pioneers the integration of Neural Descriptor Fields (NDFs) into dynamic recovery control. CADRE introduces an implicit contact-aware module that directly encodes finger–object contact geometry without requiring explicit pose estimates or point-cloud inputs. The method enables zero-shot generalization across unseen object shapes, substantially improving reinforcement learning training efficiency and convergence stability. Experiments demonstrate that CADRE significantly increases success rates in dynamic catch-and-grasp tasks and achieves robust cross-object transfer performance on previously unseen geometries.

Technology Category

• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Real-world dexterous manipulation often encounters unexpected errors and disturbances, which can lead to catastrophic failures, such as dropping the manipulated object. To address this challenge, we focus on the problem of catching a falling object while it remains within grasping range and, importantly, resetting the system to a configuration favorable for resuming the primary manipulation task. We propose Contact-Aware Dynamic Recovery (CADRE), a reinforcement learning framework that incorporates a Neural Descriptor Field (NDF)-inspired module to extract implicit contact features. Compared to methods that rely solely on object pose or point cloud input, NDFs can directly reason about finger-object correspondence and adapt to different object geometries. Our experiments show that incorporating contact features improves training efficiency, enhances convergence performance for RL training, and ultimately leads to more successful recoveries. Additionally, we demonstrate that CADRE can generalize zero-shot to unseen objects with different geometries.

Problem

• Recovering from manipulation errors using contact-aware reinforcement learning
• Catching falling objects while maintaining grasp readiness
• Generalizing recovery strategies to unseen object geometries

Innovation

• Uses Neural Descriptor Fields for contact features
• Reinforcement learning framework for dynamic recovery
• Generalizes zero-shot to unseen object geometries