🤖 AI Summary
Existing robotic manipulation datasets are often limited to successful executions or single skills, lacking coverage of anomalous behaviors and thus hindering the development of robust policy learning. To address this gap, this work introduces COFFAIL, a novel dataset collected in a real kitchen environment using a physical dual-arm robot performing coffee-making tasks. COFFAIL encompasses multimodal recordings of diverse manipulation skills executed both successfully and under various failure conditions, along with coordinated bimanual actions—all situated within a unified task context. This dataset is the first to jointly capture multi-skill execution, multiple outcome types (success and anomalies), and bimanual coordination in a realistic setting. Experimental results demonstrate that imitation learning policies trained on COFFAIL exhibit strong effectiveness and generalization capabilities.
📝 Abstract
In the context of robot learning for manipulation, curated datasets are an important resource for advancing the state of the art; however, available datasets typically only include successful executions or are focused on one particular type of skill. In this short paper, we briefly describe a dataset of various skills performed in the context of coffee preparation. The dataset, which we call COFFAIL, includes both successful and anomalous skill execution episodes collected with a physical robot in a kitchen environment, a couple of which are performed with bimanual manipulation. In addition to describing the data collection setup and the collected data, the paper illustrates the use of the data in COFFAIL to learn a robot policy using imitation learning.