ROSER: Few-Shot Robotic Sequence Retrieval for Scalable Robot Learning

2026-03-02

AI Summary

This work addresses the scarcity of task-annotated and segmented training data in robot learning by formulating data curation as a few-shot sequential retrieval problem. It introduces a task-agnostic temporal metric learning approach that, given only 3–5 examples, efficiently retrieves reusable task segments from unlabeled continuous interaction logs without requiring task-specific training. The method leverages a lightweight framework combining temporal window embeddings with metric learning to achieve high retrieval accuracy. Evaluated on benchmarks including LIBERO, DROID, and nuScenes, it significantly outperforms conventional alignment techniques, embedding-based methods, and language models, while achieving sub-millisecond inference latency and superior distributional alignment.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

A critical bottleneck in robot learning is the scarcity of task-labeled, segmented training data, despite the abundance of large-scale robotic datasets recorded as long, continuous interaction logs. Existing datasets contain vast amounts of diverse behaviors, yet remain structurally incompatible with modern learning frameworks that require cleanly segmented, task-specific trajectories. We address this data utilization crisis by formalizing robotic sequence retrieval: the task of extracting reusable, task-centric segments from unlabeled logs using only a few reference examples. We introduce ROSER, a lightweight few-shot retrieval framework that learns task-agnostic metric spaces over temporal windows, enabling accurate retrieval with as few as 3-5 demonstrations, without any task-specific training required. To validate our approach, we establish comprehensive evaluation protocols and benchmark ROSER against classical alignment methods, learned embeddings, and language model baselines across three large-scale datasets (e.g., LIBERO, DROID, and nuScenes). Our experiments demonstrate that ROSER consistently outperforms all prior methods in both accuracy and efficiency, achieving sub-millisecond per-match inference while maintaining superior distributional alignment. By reframing data curation as few-shot retrieval, ROSER provides a practical pathway to unlock underutilized robotic datasets, fundamentally improving data availability for robot learning.

Problem

• robot learning
• sequence retrieval
• few-shot learning
• data segmentation
• task-labeled data

Innovation

• few-shot retrieval
• robotic sequence segmentation
• task-agnostic metric learning
• scalable robot learning
• unlabeled trajectory extraction