LightStereo: Channel Boost Is All Your Need for Efficient 2D Cost Aggregation

2024-06-28

arXiv.org

AI Summary

To address the challenge of simultaneously achieving high accuracy, efficiency, and low computational cost in real-time stereo matching, this paper proposes a lightweight 3D cost volume modeling framework. It replaces computationally expensive 4D cost aggregation with a channel-optimized 3D cost volume and, for the first time, systematically exploits the representational capacity along the channel dimension via a multi-strategy channel enhancement mechanism—including channel-wise attention refinement and lightweight feature encoding. The framework further integrates 2D cost aggregation with an efficient spatial regularization network. Despite an extremely low computational load (22 GFLOPs, 17 ms), the model achieves competitive end-point error (EPE) on SceneFlow and ranks first on the KITTI 2015 real-time leaderboard. The core contribution lies in establishing a channel-dimension-driven paradigm for efficient 3D cost volume modeling.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We present LightStereo, a cutting-edge stereo-matching network crafted to accelerate the matching process. Departing from conventional methodologies that rely on aggregating computationally intensive 4D costs, LightStereo adopts the 3D cost volume as a lightweight alternative. While similar approaches have been explored previously, our breakthrough lies in enhancing performance through a dedicated focus on the channel dimension of the 3D cost volume, where the distribution of matching costs is encapsulated. Our exhaustive exploration has yielded plenty of strategies to amplify the capacity of the pivotal dimension, ensuring both precision and efficiency. We compare the proposed LightStereo with existing state-of-the-art methods across various benchmarks, which demonstrate its superior performance in speed, accuracy, and resource utilization. LightStereo achieves a competitive EPE metric in the SceneFlow datasets while demanding a minimum of only 22 GFLOPs and 17 ms of runtime, and ranks 1st on KITTI 2015 among real-time models. Our comprehensive analysis reveals the effect of 2D cost aggregation for stereo matching, paving the way for real-world applications of efficient stereo systems. Code will be available at https://github.com/XiandaGuo/OpenStereo.

Problem

• Efficient stereo-matching network
• Enhancing 3D cost volume channel
• Real-time performance in stereo systems

Innovation

• Focuses on channel dimension enhancement
• Uses 3D cost volume for efficiency
• Achieves superior speed and accuracy