BaryIR: Learning Multi-Source Unified Representation in Continuous Barycenter Space for Generalizable All-in-One Image Restoration

2025-05-27

AI Summary

Existing general-purpose image restoration (AIR) methods exhibit limited generalization under out-of-distribution degradations. To address this, we propose a two-level latent-space modeling framework: (1) a continuous barycentric space for degradation-invariant unified representation, and (2) source-specific orthogonal subspaces for degradation-aware semantic modeling. We introduce, for the first time, the multi-source latent optimal transport barycenter problem into image restoration, designing a continuous barycentric mapping mechanism that jointly ensures subspace orthogonality, cross-degradation representation disentanglement, and geometric consistency. Additionally, we incorporate latent-space orthogonal contrastive learning and degradation-aware dual-stream encoding. Our method achieves state-of-the-art performance on diverse synthetic and real-world degradation benchmarks, significantly improving generalization to unseen degradation types and complex realistic scenarios. The code is publicly available.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Despite remarkable advances made in all-in-one image restoration (AIR) for handling different types of degradations simultaneously, existing methods remain vulnerable to out-of-distribution degradations and images, limiting their real-world applicability. In this paper, we propose a multi-source representation learning framework BaryIR, which decomposes the latent space of multi-source degraded images into a continuous barycenter space for unified feature encoding and source-specific subspaces for specific semantic encoding. Specifically, we seek the multi-source unified representation by introducing a multi-source latent optimal transport barycenter problem, in which a continuous barycenter map is learned to transport the latent representations to the barycenter space. The transport cost is designed such that the representations from source-specific subspaces are contrasted with each other while maintaining orthogonality to those from the barycenter space. This enables BaryIR to learn compact representations with unified degradation-agnostic information from the barycenter space, as well as degradation-specific semantics from source-specific subspaces, capturing the inherent geometry of multi-source data manifold for generalizable AIR. Extensive experiments demonstrate that BaryIR achieves competitive performance compared to state-of-the-art all-in-one methods. Particularly, BaryIR exhibits superior generalization ability to real-world data and unseen degradations. The code will be publicly available at https://github.com/xl-tang3/BaryIR.

Problem

• Learning unified image restoration for diverse degradations
• Handling out-of-distribution degradations and images effectively
• Generalizing to real-world data and unseen degradations

Innovation

• Multi-source unified representation in barycenter space
• Continuous barycenter map for latent transport
• Orthogonal contrastive learning for degradation-specific semantics