Improving Low-Cost Teleoperation: Augmenting GELLO with Force

2025-07-17

AI Summary

This study addresses the absence of haptic perception in the low-cost teleoperation system GELLO—caused by its lack of force feedback—which degrades imitation learning data quality and policy performance. We present the first integration of a six-axis force sensor and real-time force feedback into GELLO’s architecture, enabling bidirectional perception of environmental resistance and closed-loop human-robot interaction. Methodologically, force signals are synchronized with motion capture and incorporated directly into imitation learning training, yielding a force-motion coupled, data-driven model, validated jointly on physical hardware and simulation. Experiments demonstrate that force augmentation significantly improves success rates in dexterous manipulation tasks (average +23.6%); user studies confirm experienced operators strongly prefer the force-feedback controller. Key contributions include: (1) the first open-source, force-feedback-enabled GELLO implementation; (2) a novel force-augmented imitation learning data collection paradigm; and (3) empirical validation of the critical role of haptic cues in enhancing generalization capability of teleoperation policies.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Interaction Techniques and Devices

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In this work we extend the low-cost GELLO teleoperation system, initially designed for joint position control, with additional force information. Our first extension is to implement force feedback, allowing users to feel resistance when interacting with the environment. Our second extension is to add force information into the data collection process and training of imitation learning models. We validate our additions by implementing these on a GELLO system with a Franka Panda arm as the follower robot, performing a user study, and comparing the performance of policies trained with and without force information on a range of simulated and real dexterous manipulation tasks. Qualitatively, users with robotics experience preferred our controller, and the addition of force inputs improved task success on the majority of tasks.

Problem

• Extend GELLO teleoperation system with force feedback
• Incorporate force data in imitation learning training
• Validate force-augmented system on manipulation tasks

Innovation

• Implemented force feedback for user resistance
• Added force data to imitation learning training
• Validated with Franka Panda arm and user study