Phase-Conditioned Imitation Learning with Autonomous Failure Recovery for Robust Deformable Object Manipulation

2026-05-28

AI Summary

This work addresses the state ambiguity problem in standard imitation learning for deformable object manipulation, where the Markov assumption fails to resolve conflicting action requirements under visually similar states, leading to an inability to autonomously recover from errors. To overcome this limitation, the authors propose a closed-loop hierarchical architecture that integrates a FiLM-conditioned ACT encoder, a hybrid impedance controller, and a tactile teleoperation interface. The system employs a multimodal phase predictor—fusing vision, force, and pose—to detect contact failures in real time and trigger recovery trajectories. Crucially, a FiLM-based conditional modulation mechanism enables a unified policy that is aware of task phases. Evaluated on T-shirt hanging and removal tasks, the approach improves success rates from 56% to 87%, significantly outperforming both unconditional and token-level conditional baselines.

Abstract

This paper presents a phase-conditioned, force-aware framework for robust deformable object manipulation. Standard imitation learning policies such as Action Chunking with Transformers (ACT) rely on a Markovian assumption at inference, causing state aliasing when visually similar observations require contradictory actions and preventing autonomous recovery from execution failures. We address this with a closed-loop hierarchical architecture. A FiLM-conditioned ACT encoder modulates feature extraction based on the current task phase, enabling a single unified policy to produce phase-specific behaviors while sharing action dynamics across phases. A multi-modal phase predictor fusing visual, force, and pose feedback estimates the phase in real time, detecting contact failures that are invisible to vision alone and autonomously triggering recovery trajectories. The system is completed by a hybrid impedance controller for compliant execution and a haptic teleoperation interface for force-aware data collection. Ablation studies show that FiLM-based modulation significantly outperforms both unconditioned and token-level conditioned baselines, and t-SNE analysis confirms that FiLM induces well-separated, phase-specific feature representations. Validated on hanging and removing a T-shirt with dual arms, the closed-loop system improves the hanging success rate from 56\% to 87\% through autonomous error recovery. Code and videos: https://leledeyuan00.github.io/phaser/

Problem

• deformable object manipulation
• imitation learning
• state aliasing
• execution failure recovery
• phase-aware control

Innovation

• Phase-Conditioned Imitation Learning
• FiLM Modulation
• Force-Aware Manipulation
• Autonomous Failure Recovery
• Deformable Object Manipulation