Semantic SLAM with Rolling-Shutter Cameras and Low-Precision INS in Outdoor Environments

2025-04-01

AI Summary

To address the low accuracy and severe drift in outdoor autonomous driving localization and mapping when fusing consumer-grade rolling-shutter cameras with low-cost inertial navigation systems (INS), this paper proposes the first real-time method integrating road-level semantic features—specifically lane markings and traffic signs—into a tightly coupled graph-optimization SLAM framework, jointly compensating for rolling-shutter distortion and long-term INS drift. Our approach unifies a semantic detection network, a rolling-shutter motion compensation model, multi-sensor (camera/IMU/wheel odometry) tight coupling in nonlinear optimization, and structural road priors as geometric constraints. Experiments demonstrate improvements of 5.35% in semantic detection recall and 2.79% in precision; relative pose error is bounded within 10 cm, and absolute positioning error remains approximately 1 m—even under challenging urban conditions—while maintaining robustness. The method significantly enhances production-grade localization performance on cost-constrained hardware.

Technology Category

• Intelligent Robots: Localization, Mapping, and Navigation
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Accurate localization and mapping in outdoor environments remains challenging when using consumer-grade hardware, particularly with rolling-shutter cameras and low-precision inertial navigation systems (INS). We present a novel semantic SLAM approach that leverages road elements such as lane boundaries, traffic signs, and road markings to enhance localization accuracy. Our system integrates real-time semantic feature detection with a graph optimization framework, effectively handling both rolling-shutter effects and INS drift. Using a practical hardware setup which consists of a rolling-shutter camera (3840*2160@30fps), IMU (100Hz), and wheel encoder (50Hz), we demonstrate significant improvements over existing methods. Compared to state-of-the-art approaches, our method achieves higher recall (up to 5.35%) and precision (up to 2.79%) in semantic element detection, while maintaining mean relative error (MRE) within 10cm and mean absolute error (MAE) around 1m. Extensive experiments in diverse urban environments demonstrate the robust performance of our system under varying lighting conditions and complex traffic scenarios, making it particularly suitable for autonomous driving applications. The proposed approach provides a practical solution for high-precision localization using affordable hardware, bridging the gap between consumer-grade sensors and production-level performance requirements.

Problem

• Enhancing localization accuracy with semantic SLAM in outdoor environments
• Handling rolling-shutter effects and INS drift in consumer-grade hardware
• Improving recall and precision in semantic element detection for autonomous driving

Innovation

• Semantic SLAM with rolling-shutter cameras
• Real-time semantic feature detection integration
• Graph optimization handles INS drift