MVOFormer: Flow-Semantic Transformer for Robust Monocular Visual Odometry

2026-06-15

AI Summary

Existing learning-based monocular visual odometry methods struggle to achieve both robustness and cross-domain generalization due to either a lack of interpretable complementary features or excessive architectural complexity. This work proposes MVOFormer, the first approach to jointly model dense optical flow and semantic information within an end-to-end Transformer architecture for monocular visual odometry. It employs a dual-branch encoder that fuses geometric motion cues from optical flow with object-level semantic priors, coupled with an iterative multimodal decoder that refines pose estimates from coarse to fine while dynamically suppressing attention to unreliable regions. Without any target-domain fine-tuning, MVOFormer significantly outperforms current frame-to-frame learning methods across multiple benchmarks—including TartanAir, KITTI, TUM-RGBD, and ETH3D-SLAM—demonstrating exceptional zero-shot generalization and robustness.

Abstract

Monocular visual odometry (MVO) is foundational to autonomous navigation and robotic localization. However, existing learning-based MVO approaches often struggle with either a lack of interpretable, complementary features or overly complex multi-stage architectures. These limitations inherently restrict their robustness and cross-domain generalization. In this work, we propose MVOFormer, a novel transformer framework for robust monocular visual odometry. Our architecture features a Flow-Semantic Dual Branch Encoder that synergizes dense geometric motion cues with object-centric semantic priors, explicitly distinguishing static structures from dynamic distractors. These representations are then fused by an Iterative Multimodal Decoder, enabling coarse-to-fine pose refinement while dynamically suppressing attention on unreliable regions. Extensive evaluations demonstrate that, without any target-domain fine-tuning, MVOFormer achieves superior zero-shot generalization and robustness, significantly outperforming prior learning-based frame-to-frame methods across diverse benchmarks including TartanAir, KITTI, TUM-RGBD, and ETH3D-SLAM.

Problem

• Monocular Visual Odometry
• Robustness
• Cross-domain Generalization
• Feature Representation
• Dynamic Distractors

Innovation

• Flow-Semantic Transformer
• Monocular Visual Odometry
• Zero-shot Generalization
• Dual Branch Encoder
• Iterative Multimodal Decoder