Versatile Demonstration Interface: Toward More Flexible Robot Demonstration Collection

2024-10-24

arXiv.org

AI Summary

Existing learning-from-demonstration approaches are largely constrained to a single teaching modality—such as teleoperation, kinesthetic teaching, or natural demonstration—limiting their ability to accommodate diverse human demonstrators’ preferences and task requirements. This paper introduces the Versatile Demonstration Interface (VDI), a unified, hardware-efficient teaching interface designed for industrial collaborative robotics. VDI is the first solution to natively support all three modalities on a single platform without requiring environment calibration. Its multimodal perception architecture fuses AprilTag-based visual tracking, six-axis force sensing, and joint encoder/external pose estimation, balancing robustness with human-centered ergonomics. An expert user study conducted at a local manufacturing innovation center validates VDI’s effectiveness across representative industrial tasks. Furthermore, the study yields multiple production-line application patterns, demonstrating practical scalability and adaptability in real-world settings.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Previous methods for Learning from Demonstration leverage several approaches for a human to teach motions to a robot, including teleoperation, kinesthetic teaching, and natural demonstrations. However, little previous work has explored more general interfaces that allow for multiple demonstration types. Given the varied preferences of human demonstrators and task characteristics, a flexible tool that enables multiple demonstration types could be crucial for broader robot skill training. In this work, we propose Versatile Demonstration Interface (VDI), an attachment for collaborative robots that simplifies the collection of three common types of demonstrations. Designed for flexible deployment in industrial settings, our tool requires no additional instrumentation of the environment. Our prototype interface captures human demonstrations through a combination of vision, force sensing, and state tracking (e.g., through the robot proprioception or AprilTag tracking). Through a user study where we deployed our prototype VDI at a local manufacturing innovation center with manufacturing experts, we demonstrated VDI in representative industrial tasks. Interactions from our study highlight the practical value of VDI's varied demonstration types, expose a range of industrial use cases for VDI, and provide insights for future tool design.

Problem

• Develops a flexible interface for robot demonstration collection.
• Supports multiple demonstration types for varied human preferences.
• Enables versatile robot skill training in industrial settings.

Innovation

• Combines vision, force sensing, state tracking
• No additional environmental instrumentation needed
• Supports multiple demonstration types flexibly