Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models

2026-08-26

AI Summary

该研究通过将2D视频基础模型SAM2转化为4D LiDAR领域的监督源，自动生成高质量的LiDAR训练数据，从而解决4D LiDAR分割中数据标注成本高且难以扩展的问题。

Abstract

Progress in 4D LiDAR segmentation is bottlenecked by data. Assigning temporally consistent labels across sparse point cloud sequences is costly and hard to scale, and every new task or domain tends to demand fresh dense annotation. This motivates a simple question of whether high-quality LiDAR training data can be produced automatically, without any human labeling. To this end, we introduce LiDAR-SAM2, a framework that turns a 2D video foundation model, SAM2, into a scalable source of supervision for the 4D LiDAR domain. On the data side, it automatically generates temporally coherent LiDAR-level labels from SAM2 video masks through multi-view projection and spatio-temporal aggregation. On the modeling side, a tailored modality interface and a two-stage learning objective adapt SAM2's video segmentation kernel to spatio-temporal LiDAR structure, so that a single click per object yields a consistent mask track across the sequence. Trained with no human LiDAR annotation, LiDAR-SAM2 produces semantic and panoptic labels on SemanticKITTI that approach the quality of full human annotation from only a few points, and models trained on these labels approach the performance of full ground-truth supervision. This positions LiDAR-SAM2 as a scalable labeling tool that substantially reduces the annotation burden for 3D and 4D scene understanding.

Problem

• 4D LiDAR
• segmentation
• data annotation
• temporal consistency

Innovation

• 4D LiDAR Segmentation
• Automatic Annotation
• Video Foundation Models
• Temporal Consistency
• Spatio-temporal Aggregation