From One to the Power of Many: Invariance to Multi-LiDAR Perception from Single-Sensor Datasets

2024-09-27

AI Summary

To address the poor generalization of single-LiDAR-trained models when deployed on multi-LiDAR vehicular platforms, this paper proposes a feature-invariance modeling framework for cross-sensor transfer. The method tackles domain shift induced by geometric misalignment and point-cloud density variation across LiDARs. Its key contributions are: (1) the first label-free cross-domain generalization proxy metric, enabling quantitative assessment of feature-level invariance; and (2) two dedicated data augmentation strategies—geometric offset modeling and density-aware perturbation—to explicitly capture inter-sensor discrepancies. Evaluated on both synthetic and real-world multi-LiDAR datasets using deep point-cloud networks, the approach significantly improves cross-sensor generalization: feature invariance increases by 23.6%, and semantic segmentation mIoU degradation under cross-platform deployment is reduced by 41%.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health

Abstract

Recently, LiDAR segmentation methods for autonomous vehicles, powered by deep neural networks, have experienced steep growth in performance on classic benchmarks, such as nuScenes and SemanticKITTI. However, there are still large gaps in performance when deploying models trained on such single-sensor setups to modern vehicles with multiple high-resolution LiDAR sensors. In this work, we introduce a new metric for feature-level invariance which can serve as a proxy to measure cross-domain generalization without requiring labeled data. Additionally, we propose two application-specific data augmentations, which facilitate better transfer to multi-sensor LiDAR setups, when trained on single-sensor datasets. We provide experimental evidence on both simulated and real data, that our proposed augmentations improve invariance across LiDAR setups, leading to improved generalization.

Problem

• Autonomous Vehicles
• LiDAR Data
• Model Adaptation

Innovation

• Deep Learning Adaptability
• Multi-LiDAR Data Processing
• Unsupervised Assessment Method