FrameSkip: Learning from Fewer but More Informative Frames in VLA Training

2026-05-13

AI Summary

This work addresses the challenge in visual-language-action (VLA) model training where uniformly sampled video frames often drown out critical manipulation moments—such as grasping or contact—with an abundance of low-variation frames. To mitigate this, the authors propose FrameSkip, a data-loading framework that dynamically selects high-information frames using an importance scoring mechanism. This score integrates motion dynamics, vision-action consistency, task-progress priors, and gripper state transitions. Without altering the model architecture or training pipeline, FrameSkip retains only 20% of the most informative frames yet boosts average success rates from 66.50% to 76.15% across three benchmarks: RoboCasa-GR1, SimplerEnv, and LIBERO, achieving a significantly improved trade-off between performance and data compression.

Abstract

Vision-Language-Action (VLA) policies are commonly trained from dense robot demonstration trajectories, often collected through teleoperation, by sampling every recorded frame as if it provided equally useful supervision. We argue that this convention creates a temporal supervision imbalance: long low-change segments dominate the training stream, while manipulation-critical transitions such as alignment, contact, grasping, and release appear only sparsely. We introduce FrameSkip, a data-layer frame selection framework that scores trajectory frames using action variation, visual-action coherence, task-progress priors, and gripper-transition preservation, then remaps training samples toward high-importance frames under a target retention ratio. Because FrameSkip operates only in the dataloader, it leaves the VLA architecture, action head, training objective, and inference procedure unchanged. Across RoboCasa-GR1, SimplerEnv, and LIBERO, FrameSkip improves the success-retention trade-off over full-frame training and simpler frame selection variants, achieving a macro-average success rate of 76.15% across the three benchmarks compared with 66.50% for full-frame training while using a compressed trajectory view that retains 20% of unique frames in the main setting.

Problem

• Vision-Language-Action
• temporal supervision imbalance
• frame selection
• robot demonstration trajectories
• manipulation-critical transitions

Innovation

• FrameSkip
• Vision-Language-Action
• frame selection
• temporal supervision imbalance
• data efficiency