Weakly and Self-Supervised Class-Agnostic Motion Prediction for Autonomous Driving

2025-09-16

AI Summary

This work addresses the strong reliance of category-agnostic motion prediction in autonomous driving on large-scale motion annotations. We propose a synergistic weakly supervised and self-supervised learning framework. Methodologically, we leverage only foreground/background or ground/non-ground segmentation masks derived from LiDAR point clouds—requiring no motion annotations—as weak supervision, and introduce a multi-frame consistency-aware robust Chamfer Distance loss to suppress erroneous motion estimates in self-supervision. Our key contributions are: (i) the first systematic integration of scene parsing cues into category-agnostic motion prediction, establishing a weakly supervised guidance paradigm for self-supervised learning; and (ii) a novel loss function that significantly improves temporal consistency and robustness. Experiments demonstrate that our method substantially outperforms existing self-supervised approaches under zero-shot or 0.1% annotation settings, with several metrics approaching those of fully supervised models—achieving an effective trade-off between annotation cost and prediction accuracy.

Technology Category

• Computer Vision: Motion & Tracking
• Machine Learning: Unsupervised & Self-Supervised Learning
• Intelligent Robots: Motion and Path Planning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Understanding motion in dynamic environments is critical for autonomous driving, thereby motivating research on class-agnostic motion prediction. In this work, we investigate weakly and self-supervised class-agnostic motion prediction from LiDAR point clouds. Outdoor scenes typically consist of mobile foregrounds and static backgrounds, allowing motion understanding to be associated with scene parsing. Based on this observation, we propose a novel weakly supervised paradigm that replaces motion annotations with fully or partially annotated (1%, 0.1%) foreground/background masks for supervision. To this end, we develop a weakly supervised approach utilizing foreground/background cues to guide the self-supervised learning of motion prediction models. Since foreground motion generally occurs in non-ground regions, non-ground/ground masks can serve as an alternative to foreground/background masks, further reducing annotation effort. Leveraging non-ground/ground cues, we propose two additional approaches: a weakly supervised method requiring fewer (0.01%) foreground/background annotations, and a self-supervised method without annotations. Furthermore, we design a Robust Consistency-aware Chamfer Distance loss that incorporates multi-frame information and robust penalty functions to suppress outliers in self-supervised learning. Experiments show that our weakly and self-supervised models outperform existing self-supervised counterparts, and our weakly supervised models even rival some supervised ones. This demonstrates that our approaches effectively balance annotation effort and performance.

Problem

• Develops class-agnostic motion prediction models
• Reduces annotation needs using weak supervision
• Leverages LiDAR point clouds for autonomous driving

Innovation

• Weakly supervised learning with minimal annotations
• Self-supervised motion prediction without annotations
• Robust consistency-aware loss function design