Tightly-Coupled Radar-Visual-Inertial Odometry

2026-03-24

AI Summary

This work proposes a tightly coupled visual–FMCW radar–inertial fusion framework to address the failure of visual-inertial odometry in dark, low-texture, or occluded environments and the sparsity and insufficient long-term accuracy of radar-inertial odometry. The method jointly optimizes visual features, FMCW radar Doppler and ranging measurements, and IMU pre-integration within an iterated extended Kalman filter, while leveraging radar ranging to assist in visual depth initialization. Experimental results demonstrate that the system significantly enhances the robustness and accuracy of state estimation under extreme conditions, including indoor and outdoor darkness, fog, and high-speed flight scenarios.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Visual-Inertial Odometry (VIO) is a staple for reliable state estimation on constrained and lightweight platforms due to its versatility and demonstrated performance. However, pertinent challenges regarding robust operation in dark, low-texture, obscured environments complicate the use of such methods. Alternatively, Frequency Modulated Continuous Wave (FMCW) radars, and by extension Radar-Inertial Odometry (RIO), offer robustness to these visual challenges, albeit at the cost of reduced information density and worse long-term accuracy. To address these limitations, this work combines the two in a tightly coupled manner, enabling the resulting method to operate robustly regardless of environmental conditions or trajectory dynamics. The proposed method fuses image features, radar Doppler measurements, and Inertial Measurement Unit (IMU) measurements within an Iterated Extended Kalman Filter (IEKF) in real-time, with radar range data augmenting the visual feature depth initialization. The method is evaluated through flight experiments conducted in both indoor and outdoor environments, as well as through challenges to both exteroceptive modalities (such as darkness, fog, or fast flight), thoroughly demonstrating its robustness. The implementation of the proposed method is available at: https://github.com/ntnu-arl/radvio .

Problem

• Visual-Inertial Odometry
• Radar-Inertial Odometry
• robust state estimation
• degraded visual environments
• sensor fusion

Innovation

• Radar-Visual-Inertial Odometry
• Tightly-Coupled Fusion
• FMCW Radar
• Depth Initialization
• Iterated Extended Kalman Filter