Improving Optical Flow and Stereo Depth Estimation by Leveraging Uncertainty-Based Learning Difficulties

2025-05-31

AI Summary

Traditional optical flow and stereo depth estimation methods employ uniform pixel-wise losses, ignoring inherent variations in per-pixel learning difficulty. To address this, we propose an uncertainty-aware adaptive training framework that introduces two novel loss components: Difficulty Balancing (DB) loss and Occlusion-Avoidance (OA) loss, operating in synergy. Our method is the first to jointly leverage error confidence maps—generated in an uncertainty-driven manner—and cycle-consistency constraints for dynamic identification and weighted optimization of hard samples. The framework comprises four key elements: uncertainty modeling, error-driven confidence map generation, difficulty-weighted loss formulation, and end-to-end joint network optimization. Evaluated on multiple mainstream benchmarks, our approach achieves 8.2% and 7.6% reductions in optical flow and stereo depth estimation errors, respectively, demonstrating significant improvements in accuracy, generalization, and robustness.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Conventional training for optical flow and stereo depth models typically employs a uniform loss function across all pixels. However, this one-size-fits-all approach often overlooks the significant variations in learning difficulty among individual pixels and contextual regions. This paper investigates the uncertainty-based confidence maps which capture these spatially varying learning difficulties and introduces tailored solutions to address them. We first present the Difficulty Balancing (DB) loss, which utilizes an error-based confidence measure to encourage the network to focus more on challenging pixels and regions. Moreover, we identify that some difficult pixels and regions are affected by occlusions, resulting from the inherently ill-posed matching problem in the absence of real correspondences. To address this, we propose the Occlusion Avoiding (OA) loss, designed to guide the network into cycle consistency-based confident regions, where feature matching is more reliable. By combining the DB and OA losses, we effectively manage various types of challenging pixels and regions during training. Experiments on both optical flow and stereo depth tasks consistently demonstrate significant performance improvements when applying our proposed combination of the DB and OA losses.

Problem

• Addresses non-uniform learning difficulty in optical flow and stereo depth estimation
• Proposes Difficulty Balancing loss to focus on challenging pixels and regions
• Introduces Occlusion Avoiding loss to handle occlusion-affected unreliable matches

Innovation

• Uses error-based confidence for difficulty balancing
• Introduces occlusion avoiding loss for consistency
• Combines DB and OA losses for performance