MUSE: Multimodal Uncertainty Quantification of State Estimation

2026-05-17

AI Summary

This work addresses the challenges of uncertainty quantification, unreliable confidence estimation, and failure detection in visual-inertial odometry under heteroscedastic and multimodal conditions. To this end, we propose the first real-time learning framework based on the Mamba architecture for jointly modeling asynchronous multimodal sensor streams and estimating localization uncertainty. Our approach innovatively leverages Mamba’s powerful sequential modeling capabilities within multimodal state estimation, integrating a multi-source fusion strategy with a learning-driven uncertainty calibration mechanism to achieve efficient and reliable confidence quantification. Extensive experiments demonstrate that our method significantly outperforms existing approaches on both public and self-collected datasets, achieving notable advances in robustness and reliability, while ablation studies confirm the effectiveness of our core design choices.

Abstract

Accurate visual state estimation has been a central topic in robotics with a wide range of applications in robot navigation, autonomous driving, and autonomous flight. Recent advances in robot perception have led to significant improvements in the accuracy and robustness of state estimation, yet a fundamental challenge remains in how to quantify and calibrate its precision, i.e., how confident we are in an estimate and whether failures can be detected. This issue is particularly pronounced in visual-inertial odometry (VIO), where the heteroscedastic and multimodal nature of the problem makes uncertainty quantification especially difficult. This paper introduces MUSE (Multimodal Uncertainty Quantification of State Estimation), a novel real-time learning-based framework that leverages the strong and efficient sequential modeling capacity of Mamba to estimate localization uncertainty from multiple asynchronous sensor streams. Experiments on both public and in-house datasets demonstrate that MUSE achieves superior reliability and robustness compared to existing uncertainty quantification methods, and ablation studies justify the benefits of its key design choices.

Problem

• Uncertainty Quantification
• State Estimation
• Visual-Inertial Odometry
• Multimodal
• Heteroscedasticity

Innovation

• Uncertainty Quantification
• Visual-Inertial Odometry
• Mamba Architecture
• Multimodal Sensor Fusion
• State Estimation