DuCos: Duality Constrained Depth Super-Resolution via Foundation Model

2025-03-06

AI Summary

To address weak cross-scene generalization, geometric distortions, and edge blurring in deep super-resolution, this paper proposes DuCos—a dual-consistency framework. First, it incorporates geometric alignment prompts generated by foundation models into a Lagrangian dual constraint, thereby formulating a structured prior-guided optimization objective. Second, it introduces two synergistic modules: Correlation Fusion (CF) for geometric consistency and Gradient Regulation (GR) for edge fidelity. Additionally, edge-aware gradient regularization and multi-scale feature fusion are integrated to enhance structural preservation. Extensive experiments demonstrate that DuCos achieves state-of-the-art performance across multiple benchmarks, reducing RMSE by 18.7% compared to prior methods. It further exhibits superior noise robustness and cross-domain generalization capability. The source code and pre-trained models will be publicly released.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Neural Architectures and Foundation Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We introduce DuCos, a novel depth super-resolution framework grounded in Lagrangian duality theory, offering a flexible integration of multiple constraints and reconstruction objectives to enhance accuracy and robustness. Our DuCos is the first to significantly improve generalization across diverse scenarios with foundation models as prompts. The prompt design consists of two key components: Correlative Fusion (CF) and Gradient Regulation (GR). CF facilitates precise geometric alignment and effective fusion between prompt and depth features, while GR refines depth predictions by enforcing consistency with sharp-edged depth maps derived from foundation models. Crucially, these prompts are seamlessly embedded into the Lagrangian constraint term, forming a synergistic and principled framework. Extensive experiments demonstrate that DuCos outperforms existing state-of-the-art methods, achieving superior accuracy, robustness, and generalization. The source codes and pre-trained models will be publicly available.

Problem

• Enhances depth super-resolution accuracy and robustness
• Improves generalization across diverse scenarios using foundation models
• Integrates geometric alignment and gradient regulation for precise depth predictions

Innovation

• Lagrangian duality theory for depth super-resolution
• Correlative Fusion and Gradient Regulation prompts
• Seamless integration of foundation model constraints