TSD: A Physics-Inspired Trajectory Saliency Detector for Efficient Imitation Learning

2026-06-22

AI Summary

This work addresses the scarcity and high acquisition cost of high-quality trajectory data in robotic imitation learning by proposing a training-free, plug-and-play trajectory saliency detection framework. Leveraging trajectory heterogeneity, it categorizes actions into transitional, fine, and agile segments, prioritizing fine and agile portions that are critical to task success. The study introduces, for the first time, a physics-inspired notion of saliency: spatial entropy quantifies fine manipulation, while centripetal acceleration identifies agile maneuvers, together forming a physics-driven saliency detection mechanism. Experimental results in both simulation and real-world environments demonstrate that using only 75% of the original trajectory data—selected via this saliency framework—achieves comparable or even superior performance, substantially improving data efficiency.

Abstract

For imitation learning in robotic manipulation, high data collection costs result in the scarcity of high quality data. In this paper, we leverage the inherent heterogeneity of trajectories to address this challenge. Based on our observations of manipulation tasks, we categorize motions into transitional, precise, and agile types, defining the latter two as trajectory saliency due to their criticality to task success in contrast to the prevalent but less relevant transitional motions. Therefore, we propose the Trajectory Saliency Detector (TSD), a training-free and plug-and-play framework to identify trajectory saliency. TSD employs two physically-grounded metrics: spatial entropy to capture fine-grained manipulation and centripetal acceleration to detect agile maneuvering. We further leverage TSD to develop a dataset compression method that reduces training costs and a dataset expansion strategy that improves data collection efficiency. Extensive experiments in both simulation and real-world settings demonstrate that models trained on TSD-condensed datasets achieve comparable or even superior performance with 25% less data on average. These results validate the effectiveness of our dataset compression and expansion strategies, thereby confirming the utility of TSD. Consequently, TSD offers a scalable and cost-effective pathway to synthesize information-dense datasets for efficient robot learning. Project page: https://trajectory-saliency-detector.github.io/trajectory-saliency-detector/

Problem

• imitation learning
• data scarcity
• robotic manipulation
• trajectory saliency
• data efficiency

Innovation

• Trajectory Saliency Detection
• Physics-Inspired Metrics
• Imitation Learning
• Dataset Compression
• Spatial Entropy