FACTR: Force-Attending Curriculum Training for Contact-Rich Policy Learning

2025-02-24

AI Summary

This work addresses the underutilization of force feedback and poor policy generalization in contact-intensive robotic manipulation. Methodologically, we propose a force-aware visual degradation curriculum learning framework: (1) a low-cost bilateral teleoperation system is built to collect force-augmented demonstration data; (2) a progressive visual noise degradation curriculum is designed to guide a Transformer-based architecture to prioritize force modality over visual inputs, mitigating visual overfitting; and (3) force signals are explicitly modeled as dominant in a multimodal attention mechanism. Our key contribution is the first formulation of force as the core driver of curriculum learning—establishing its primacy in contact policy acquisition. Experiments demonstrate a 43% performance improvement over non-curriculum baselines on unseen-object generalization tasks, significantly enhancing robustness and cross-object adaptability in contact-rich manipulation.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Many contact-rich tasks humans perform, such as box pickup or rolling dough, rely on force feedback for reliable execution. However, this force information, which is readily available in most robot arms, is not commonly used in teleoperation and policy learning. Consequently, robot behavior is often limited to quasi-static kinematic tasks that do not require intricate force-feedback. In this paper, we first present a low-cost, intuitive, bilateral teleoperation setup that relays external forces of the follower arm back to the teacher arm, facilitating data collection for complex, contact-rich tasks. We then introduce FACTR, a policy learning method that employs a curriculum which corrupts the visual input with decreasing intensity throughout training. The curriculum prevents our transformer-based policy from over-fitting to the visual input and guides the policy to properly attend to the force modality. We demonstrate that by fully utilizing the force information, our method significantly improves generalization to unseen objects by 43% compared to baseline approaches without a curriculum. Video results and instructions at https://jasonjzliu.com/factr/

Problem

• Utilizes force feedback in robot tasks
• Improves policy learning with curriculum training
• Enhances generalization to unseen objects

Innovation

• Bilateral teleoperation relays force feedback
• Curriculum training corrupts visual input
• Transformer-based policy utilizes force modality