AirSLAM: An Efficient and Illumination-Robust Point-Line Visual SLAM System

2024-08-07

IEEE Transactions on robotics

AI Summary

To address the degradation of robustness and efficiency in visual SLAM under short- and long-term illumination variations, this paper proposes a point-line joint, illumination-robust SLAM system. Methodologically, it introduces: (1) the first unified CNN architecture that simultaneously extracts keypoints and structural line features; (2) a lightweight relocalization module leveraging prebuilt map points, lines, and structural graphs for rapid pose recovery; and (3) an end-to-end coupled optimization framework integrating front-end geometric constraints with back-end graph optimization (g2o/Ceres). The system is accelerated via TensorRT, achieving 73 Hz on PC and 40 Hz on embedded platforms. Evaluated on multiple illumination-challenging datasets, it significantly outperforms state-of-the-art methods. The source code is publicly available.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In this paper, we present an efficient visual SLAM system designed to tackle both short-term and long-term illumination challenges. Our system adopts a hybrid approach that combines deep learning techniques for feature detection and matching with traditional backend optimization methods. Specifically, we propose a unified convolutional neural network (CNN) that simultaneously extracts keypoints and structural lines. These features are then associated, matched, triangulated, and optimized in a coupled manner. Additionally, we introduce a lightweight relocalization pipeline that reuses the built map, where keypoints, lines, and a structure graph are used to match the query frame with the map. To enhance the applicability of the proposed system to real-world robots, we deploy and accelerate the feature detection and matching networks using C++ and NVIDIA TensorRT. Extensive experiments conducted on various datasets demonstrate that our system outperforms other state-of-the-art visual SLAM systems in illumination-challenging environments. Efficiency evaluations show that our system can run at a rate of 73Hz on a PC and 40Hz on an embedded platform. Our implementation is open-sourced: https://github.com/sair-lab/AirSLAM.

Problem

• Develops illumination-robust visual SLAM system
• Combines deep learning with backend optimization
• Enhances efficiency and relocalization for robotics

Innovation

• Hybrid deep learning and optimization
• Unified CNN for feature extraction
• Lightweight relocalization pipeline