Cross-Spectral Stereo Inertial Odometry

2026-06-29

AI Summary

This work addresses the limitations of conventional monospectral visual-inertial odometry, which is prone to failure under illumination-degraded conditions and suffers from spectral redundancy. The authors propose an asynchronous, real-time tightly coupled visible-thermal-inertial fusion system that decouples high-latency depth matching from high-frequency state estimation. A spectral-aware weighting mechanism, grounded in photometric entropy and thermal noise, dynamically modulates multimodal reliance. Furthermore, a seamless pipeline integrating non-uniformity correction (NUC) for thermal imagery with deep-learning-based feature matching effectively mitigates inter-modal appearance discrepancies and environmental disturbances. Experimental results demonstrate that the proposed method achieves higher accuracy under normal lighting and maintains robust tracking in visually degraded scenarios, significantly outperforming existing monospectral systems.

Abstract

Standard stereo VIO focuses exclusively on the benefit of metric scale via single-spectrum baselines, often overlooking the risks of spectral redundancy. This structural limitation leads to correlated failures, where both sensors simultaneously fail in degraded environments that affect their shared spectrum. Leveraging a cross-spectral system presents a complementary solution to this issue, yet the significant appearance gap between modalities renders standard matching ineffective. Existing deep learning-based matchers, while effective, introduce inference latencies that violate real-time constraints. To bridge this gap, we present an asynchronous real-time cross-spectral visual-thermal-inertial (VTI) system that temporally decouples high-latency deep matching from high-rate state estimation. Our architecture incorporates a spectral-aware weighting scheme that dynamically balances modality reliance based on photometric entropy and thermal noise, ensuring robustness against both abrupt lighting changes and thermal artifacts. Furthermore, we introduce a seamless handling mechanism for thermal Non-uniformity Correction (NUC) to maintain tracking continuity. Extensive experiments across diverse scenarios confirm that our system overcomes spectral redundancy, yielding superior accuracy in nominal daylight while ensuring robustness in visually degraded environments. We will open source our code and data: https://github.com/seungsang07/cross-spectral-stereo-inertial-odometry

Problem

• cross-spectral
• visual-inertial odometry
• spectral redundancy
• real-time matching
• modality gap

Innovation

• cross-spectral
• visual-thermal-inertial odometry
• asynchronous matching
• spectral-aware weighting
• NUC handling