MoCha-Stereo: Motif Channel Attention Network for Stereo Matching

2024-04-10

Computer Vision and Pattern Recognition

AI Summary

Existing stereo matching methods often lose geometric structural information during feature extraction, leading to erroneous disparity estimation at object boundaries. To address this, we propose an end-to-end network incorporating motif-channel attention. First, we construct a Motif Channel Correlation Volume (MCCV) to explicitly model cross-view geometric structural consistency. Second, we design a Reconstruction Error Motif Penalty (REMP) module that fuses frequency-domain error features to refine disparity prediction. Third, we introduce motif-channel projection and channel-wise geometric prior modeling, jointly optimized with a frequency-domain error-guided loss for high-precision, full-resolution disparity regression. Our method achieves state-of-the-art performance on both KITTI-2015 and KITTI-2012 benchmarks, ranking first on the official leaderboards. Moreover, it demonstrates strong generalization capability in multi-view stereo reconstruction tasks.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Learning-based stereo matching techniques have made significant progress. However, existing methods inevitably lose geometrical structure information during the feature channel generation process, resulting in edge detail mis-matches. In this paper, the Motif Channel Attention Stereo Matching Network (MoCha-Stereo) is designed to address this problem. We provide the Motif Channel Correlation Volume (MCCV) to determine more accurate edge matching costs. MCCV is achieved by projecting motif channels, which capture common geometric structures in feature channels, onto feature maps and cost volumes. In addition, edge variations in the reconstruction error map also affect details matching, we propose the Reconstruction Error Motif Penalty (REMP) module to further refine the full-resolution disparity estimation. REMP integrates the frequency information of typical channel features from the re-construction error. MoCha-Stereo ranks 1st on the KITTI-2015 and KITTI-2012 Reflective leaderboards. Our structure also shows excellent performance in Multi- View Stereo.

Problem

• Addresses edge detail mismatches in stereo matching
• Improves geometric structure preservation in feature channels
• Refines full-resolution disparity estimation using reconstruction errors

Innovation

• Motif Channel Correlation Volume for edge matching
• Reconstruction Error Motif Penalty module
• Projecting motif channels onto feature maps