DenVisCoM: Dense Vision Correspondence Mamba for Efficient and Real-time Optical Flow and Stereo Estimation

2026-02-02

AI Summary

This work proposes a lightweight hybrid architecture that integrates DenVisCoM Mamba modules with Transformer attention mechanisms to address the challenge of simultaneously achieving real-time performance, memory efficiency, and high accuracy in optical flow and disparity estimation. By introducing the Mamba structure—originally developed for sequence modeling—into dense visual correspondence tasks for the first time, the method establishes a unified multi-task perception framework. This design significantly reduces computational overhead and memory consumption while maintaining competitive accuracy. Extensive experiments demonstrate that the model enables efficient, real-time joint estimation of optical flow and disparity across multiple benchmark datasets, achieving a favorable balance between speed and performance.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In this work, we propose a novel Mamba block DenVisCoM, as well as a novel hybrid architecture specifically tailored for accurate and real-time estimation of optical flow and disparity estimation. Given that such multi-view geometry and motion tasks are fundamentally related, we propose a unified architecture to tackle them jointly. Specifically, the proposed hybrid architecture is based on DenVisCoM and a Transformer-based attention block that efficiently addresses real-time inference, memory footprint, and accuracy at the same time for joint estimation of motion and 3D dense perception tasks. We extensively analyze the benchmark trade-off of accuracy and real-time processing on a large number of datasets. Our experimental results and related analysis suggest that our proposed model can accurately estimate optical flow and disparity estimation in real time. All models and associated code are available at https://github.com/vimstereo/DenVisCoM.

Problem

• optical flow
• stereo estimation
• real-time
• dense correspondence
• multi-view geometry

Innovation

• Mamba
• Optical Flow
• Stereo Estimation
• Real-time Inference
• Hybrid Architecture