Panoramic Direct LiDAR-assisted Visual Odometry

2024-09-14

arXiv.org

AI Summary

To address the poor robustness of monocular pinhole-camera visual odometry—caused by narrow field-of-view, texture scarcity, and motion blur—this paper proposes a tightly coupled direct-method odometry integrating 360° panoramic vision and sparse LiDAR. Methodologically, it achieves, for the first time, pixel-level direct alignment between panoramic images and panoramic LiDAR point clouds; introduces multi-view geometric constraints within a single frame to overcome limitations of conventional single-view temporal modeling; and formulates photometric error optimization on the spherical manifold for pose estimation. Its key contributions include a unified framework for joint calibration, spatiotemporal synchronization, and direct photometric modeling. The method significantly improves accuracy and robustness in low-texture and high-speed motion scenarios. Evaluated on public benchmarks, it reduces translational and rotational errors by 32% and 27%, respectively, outperforming current state-of-the-art approaches.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Enhancing visual odometry by exploiting sparse depth measurements from LiDAR is a promising solution for improving tracking accuracy of an odometry. Most existing works utilize a monocular pinhole camera, yet could suffer from poor robustness due to less available information from limited field-of-view (FOV). This paper proposes a panoramic direct LiDAR-assisted visual odometry, which fully associates the 360-degree FOV LiDAR points with the 360-degree FOV panoramic image datas. 360-degree FOV panoramic images can provide more available information, which can compensate inaccurate pose estimation caused by insufficient texture or motion blur from a single view. In addition to constraints between a specific view at different times, constraints can also be built between different views at the same moment. Experimental results on public datasets demonstrate the benefit of large FOV of our panoramic direct LiDAR-assisted visual odometry to state-of-the-art approaches.

Problem

• Enhancing visual odometry accuracy with LiDAR depth measurements
• Addressing poor robustness from limited field-of-view cameras
• Integrating 360-degree panoramic images with LiDAR data

Innovation

• Panoramic 360-degree FOV LiDAR integration
• Direct association of LiDAR points with panoramic images
• Multi-view constraints for enhanced pose estimation