Diffusing Trajectory Optimization Problems for Recovery During Multi-Finger Manipulation

2025-10-08

AI Summary

Multi-fingered dexterous hands frequently suffer task interruptions during tool manipulation due to external disturbances, necessitating efficient autonomous recovery mechanisms. Method: We propose a novel approach integrating diffusion models with trajectory optimization: state recovery is formulated as distribution-constrained projection onto feasible trajectories; the full trajectory parameterization—including constraints, goal states, and initial conditions—is directly modeled via diffusion; and contact-aware motion planning is embedded to generate high-success-rate, contact-rich recovery motions. Contribution/Results: Our method avoids error accumulation inherent in conventional staged planning. Evaluated on a real-robot screwdriver rotation task, it achieves a 96% recovery success rate—the first method enabling proactive recovery without triggering task failure. It significantly enhances online robustness and execution efficiency while maintaining physical feasibility and contact fidelity.

Technology Category

• Intelligent Robots: State Estimation
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multi-fingered hands are emerging as powerful platforms for performing fine manipulation tasks, including tool use. However, environmental perturbations or execution errors can impede task performance, motivating the use of recovery behaviors that enable normal task execution to resume. In this work, we take advantage of recent advances in diffusion models to construct a framework that autonomously identifies when recovery is necessary and optimizes contact-rich trajectories to recover. We use a diffusion model trained on the task to estimate when states are not conducive to task execution, framed as an out-of-distribution detection problem. We then use diffusion sampling to project these states in-distribution and use trajectory optimization to plan contact-rich recovery trajectories. We also propose a novel diffusion-based approach that distills this process to efficiently diffuse the full parameterization, including constraints, goal state, and initialization, of the recovery trajectory optimization problem, saving time during online execution. We compare our method to a reinforcement learning baseline and other methods that do not explicitly plan contact interactions, including on a hardware screwdriver-turning task where we show that recovering using our method improves task performance by 96% and that ours is the only method evaluated that can attempt recovery without causing catastrophic task failure. Videos can be found at https://dtourrecovery.github.io/.

Problem

• Autonomously detects when robotic manipulation requires recovery
• Optimizes contact-rich trajectories to resume task execution
• Efficiently plans recovery paths using diffusion-based trajectory optimization

Innovation

• Diffusion models detect out-of-distribution manipulation states
• Trajectory optimization plans contact-rich recovery paths
• Efficient diffusion sampling distills full problem parameterization online