Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation

2025-04-19

AI Summary

To address the redundancy and inefficiency arising from training separate specialized models for diverse image degradations in image restoration, this paper proposes a unified single-model framework. Methodologically, we design a lightweight degradation-aware embedding module, integrated with subspace-gated reweighting and spatial-frequency dual-path attention fusion—enabling, for the first time, degradation-adaptive modeling and frequency-domain detail enhancement without increasing model parameters. The framework handles multiple degradations—including blur, noise, and compression—within a single architecture, requiring no auxiliary modules or large language models. Extensive experiments demonstrate state-of-the-art performance across comprehensive image restoration benchmarks, while reducing model parameters by 82% and computational cost by 85% compared to existing multi-model approaches.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning on the Edge & Model Compression
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

Restoring any degraded image efficiently via just one model has become increasingly significant and impactful, especially with the proliferation of mobile devices. Traditional solutions typically involve training dedicated models per degradation, resulting in inefficiency and redundancy. More recent approaches either introduce additional modules to learn visual prompts, significantly increasing model size, or incorporate cross-modal transfer from large language models trained on vast datasets, adding complexity to the system architecture. In contrast, our approach, termed AnyIR, takes a unified path that leverages inherent similarity across various degradations to enable both efficient and comprehensive restoration through a joint embedding mechanism, without scaling up the model or relying on large language models.Specifically, we examine the sub-latent space of each input, identifying key components and reweighting them first in a gated manner. To fuse the intrinsic degradation awareness and the contextualized attention, a spatial-frequency parallel fusion strategy is proposed for enhancing spatial-aware local-global interactions and enriching the restoration details from the frequency perspective. Extensive benchmarking in the all-in-one restoration setting confirms AnyIR's SOTA performance, reducing model complexity by around 82% in parameters and 85% in FLOPs. Our code will be available at our Project page (https://amazingren.github.io/AnyIR/)

Problem

• Efficient restoration of any degraded image using one model
• Reducing model complexity and avoiding dedicated per-degradation models
• Enhancing restoration via spatial-frequency fusion without large language models

Innovation

• Unified joint embedding for efficient restoration
• Gated reweighting in sub-latent space
• Spatial-frequency parallel fusion strategy