IGEV++: Iterative Multi-range Geometry Encoding Volumes for Stereo Matching

2024-09-01

arXiv.org

AI Summary

Stereo matching remains challenging in ill-posed regions—such as occlusions and textureless areas—and under large disparities (up to 768 px), where ambiguity and estimation instability persist. To address these issues, we propose an iterative multi-scale geometric encoding volume (GEV) architecture, introducing the Multi-Range Geometric Encoding Volume (MGEV). Our method integrates adaptive patch-based matching, selective multi-granularity geometric feature fusion, and a ConvGRU-driven iterative refinement mechanism to jointly model coarse- and fine-grained geometric cues and enable efficient disparity optimization. Evaluated on Scene Flow, our approach achieves state-of-the-art performance across the full disparity range. On Middlebury, it reduces the Bad 2.0 error to 3.23%, outperforming RAFT-Stereo and GMStreos by 31.9% and 54.8%, respectively. Moreover, its real-time variant surpasses all published real-time methods on KITTI.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Stereo matching is a core component in many computer vision and robotics systems. Despite significant advances over the last decade, handling matching ambiguities in ill-posed regions and large disparities remains an open challenge. In this paper, we propose a new deep network architecture, called IGEV++, for stereo matching. The proposed IGEV++ builds Multi-range Geometry Encoding Volumes (MGEV) that encode coarse-grained geometry information for ill-posed regions and large disparities and fine-grained geometry information for details and small disparities. To construct MGEV, we introduce an adaptive patch matching module that efficiently and effectively computes matching costs for large disparity ranges and/or ill-posed regions. We further propose a selective geometry feature fusion module to adaptively fuse multi-range and multi-granularity geometry features in MGEV. We then index the fused geometry features and input them to ConvGRUs to iteratively update the disparity map. MGEV allows to efficiently handle large disparities and ill-posed regions, such as occlusions and textureless regions, and enjoys rapid convergence during iterations. Our IGEV++ achieves the best performance on the Scene Flow test set across all disparity ranges, up to 768px. Our IGEV++ also achieves state-of-the-art accuracy on the Middlebury, ETH3D, KITTI 2012, and 2015 benchmarks. Specifically, IGEV++ achieves a 3.23% 2-pixel outlier rate (Bad 2.0) on the large disparity benchmark, Middlebury, representing error reductions of 31.9% and 54.8% compared to RAFT-Stereo and GMStereo, respectively. We also present a real-time version of IGEV++ that achieves the best performance among all published real-time methods on the KITTI benchmarks. The code is publicly available at https://github.com/gangweiX/IGEV-plusplus

Problem

• Handles matching ambiguities in ill-posed stereo regions
• Addresses large disparity challenges in stereo matching
• Improves accuracy in occlusions and textureless areas

Innovation

• Multi-range Geometry Encoding Volumes for stereo matching
• Adaptive patch matching module for large disparities
• Selective geometry feature fusion for multi-granularity