Local Motion Matters: A Deconstruct-Recompose Paradigm for Reinforcement Learning Pre-training from Videos

2026-07-01

AI Summary

This work addresses the limitation of existing video-based reinforcement learning pretraining methods, which model agents as monolithic entities, resulting in motion representations tightly coupled with morphology and poor cross-domain transferability. To overcome this, the authors propose a “decompose-and-recompose” paradigm: first extracting inter-frame motions of local keypoints from videos as atomic actions, then recomposing them into global representations via learnable Motion Aggregation Tokens (MATs) and a dual-attention encoder, augmented with downstream adapters to accelerate policy learning. This approach achieves the first decoupled modeling of local motion, yielding morphology-agnostic, transferable motion representations. Experiments across diverse robotic control tasks demonstrate substantial improvements in sample efficiency and performance, validating the efficacy of local motion representations for cross-domain transfer.

Abstract

Pre-training on large-scale videos to improve reinforcement learning efficiency is promising yet remains challenging. Existing methods typically treat the agent as an indivisible entity, modeling motion patterns globally. Such global modeling is tightly coupled with the morphology, hindering transfer across domains. In contrast, despite the vast disparity in global motions, the local components exhibit similar motion patterns across different agents. Building on this insight, we propose a novel Deconstruct-Recompose Paradigm (DRP) for learning transferable local motion representations. Specifically, in the Deconstruct phase, we identify multiple local points and track their frame-wise motions, defining each as an Atomic Action. We introduce a Dual-Attention Encoder (DAE) to learn local motion representations from these Atomic Actions, capturing their spatiotemporal relationships. In the Recompose phase, we compose local motion representations with a learnable Motion Aggregation Token [MAT] via latent dynamics model learning. Additionally, an adapter bridges local motion and downstream action-specific dynamics to accelerate policy learning. Extensive experiments demonstrate that our method effectively transfers to diverse robotic control and manipulation tasks, significantly improving sample efficiency and performance.

Problem

• reinforcement learning
• video pre-training
• motion representation
• transfer learning
• local motion

Innovation

• Deconstruct-Recompose Paradigm
• Atomic Action
• Dual-Attention Encoder
• Motion Aggregation Token
• video pre-training