Pose Optimization for Autonomous Driving Datasets using Neural Rendering Models

2025-04-22

AI Summary

Autonomous driving public datasets often suffer from inaccurate sensor poses and calibration parameters, leading to distorted model evaluation. To address this, we propose the first end-to-end unsupervised joint pose optimization method based on neural radiance fields (NeRF), requiring no ground-truth labels. Our approach formulates a multi-objective optimization objective combining reprojection error, novel-view synthesis quality, and geometric consistency. It integrates differentiable rendering, multi-view geometric constraints, and L-BFGS-based nonlinear optimization. Evaluated on mainstream benchmarks—including nuScenes and KITTI—our method reduces reprojection error by 32%–47% and improves PSNR for novel-view synthesis by an average of 5.8 dB. These gains significantly enhance the reliability and generalizability of dataset benchmarks. The source code and optimized poses are publicly released.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Autonomous driving systems rely on accurate perception and localization of the ego car to ensure safety and reliability in challenging real-world driving scenarios. Public datasets play a vital role in benchmarking and guiding advancement in research by providing standardized resources for model development and evaluation. However, potential inaccuracies in sensor calibration and vehicle poses within these datasets can lead to erroneous evaluations of downstream tasks, adversely impacting the reliability and performance of the autonomous systems. To address this challenge, we propose a robust optimization method based on Neural Radiance Fields (NeRF) to refine sensor poses and calibration parameters, enhancing the integrity of dataset benchmarks. To validate improvement in accuracy of our optimized poses without ground truth, we present a thorough evaluation process, relying on reprojection metrics, Novel View Synthesis rendering quality, and geometric alignment. We demonstrate that our method achieves significant improvements in sensor pose accuracy. By optimizing these critical parameters, our approach not only improves the utility of existing datasets but also paves the way for more reliable autonomous driving models. To foster continued progress in this field, we make the optimized sensor poses publicly available, providing a valuable resource for the research community.

Problem

• Optimizing sensor poses in autonomous driving datasets
• Addressing inaccuracies in sensor calibration and vehicle poses
• Enhancing dataset reliability for autonomous driving models

Innovation

• NeRF-based pose optimization for datasets
• Reprojection metrics for accuracy validation
• Public release of optimized sensor poses