Synthetic-to-Real Translation for Class-Agnostic Motion Prediction

2026-07-07

AI Summary

This work addresses the challenges of high annotation costs for motion labels in real-world scenarios and performance degradation in motion prediction due to domain shift between synthetic and real data. To this end, the authors propose a dual-module transfer framework that integrates object-aware and object-assisted components, combining objectness-aware prior modeling, domain-invariant feature learning, and motion label denoising. Leveraging physics-based 4D LiDAR synthesis, they construct Motion4D—the first synthetic dataset specifically designed for motion prediction. Experimental results demonstrate that the proposed approach substantially narrows the domain gap between synthetic and real data, achieving robust and superior motion prediction performance in real-world settings.

Abstract

Motion understanding is critical for ensuring safety and robustness in autonomous driving systems, driving increasing interest in motion prediction. A key challenge in this domain is the high cost associated with acquiring real-world motion labels. It is therefore ideal if we could transfer motion knowledge from synthetic data to real data. In this context, we explore the potential of synthetic-to-real translation for motion prediction (SRMP). However, the most used naive motion regression methods are notably sensitive to the synthetic-to-real domain shift, resulting in unreliable knowledge translation. To address this, we propose a novel approach integrating a motion knowledge translation framework with two key components: (1) objectness-aware motion prediction, which explicitly models the joint distribution of motion patterns and objectness priors to improve domain-invariant feature learning, and (2) objectness-aided motion enhancement, a motion label refinement mechanism that leverages learned objectness priors to filter motion noise. Furthermore, we present a physically-based pipeline for generating Motion4D, the first synthetic 4D LiDAR dataset tailored for SRMP research, addressing the lack of synthetic motion datasets. Experimental results demonstrate that our approach effectively bridges the domain gaps and yields superior performance on real scenes.

Problem

• synthetic-to-real translation
• motion prediction
• domain shift
• autonomous driving
• LiDAR

Innovation

• synthetic-to-real translation
• motion prediction
• objectness-aware
• domain adaptation
• 4D LiDAR dataset