Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration

2025-12-11

AI Summary

Existing methods for unified image restoration under diverse degradations (e.g., noise, blur, adverse weather) rely on complex architectures—such as Mixture-of-Experts (MoE) or diffusion models—and require precise degradation priors, resulting in high computational cost and poor deployability. To address this, we propose SymUNet, a symmetric U-Net that explicitly models and propagates degradation information across scales via scale-aligned feature aggregation and cross-layer feature propagation, eliminating the need for expert modules or diffusion-based priors. Furthermore, we introduce SE-SymUNet, a CLIP-semantic-enhanced variant: its frozen CLIP backbone extracts degradation-relevant semantics, which are injected via lightweight cross-attention. Extensive experiments demonstrate that both SymUNet and SE-SymUNet surpass state-of-the-art methods across multiple benchmarks—achieving superior performance with significantly fewer parameters and faster inference speed. Code is publicly available.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Mixture of Experts (MoE)
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

All-in-one image restoration aims to handle diverse degradations (e.g., noise, blur, adverse weather) within a unified framework, yet existing methods increasingly rely on complex architectures (e.g., Mixture-of-Experts, diffusion models) and elaborate degradation prompt strategies. In this work, we reveal a critical insight: well-crafted feature extraction inherently encodes degradation-carrying information, and a symmetric U-Net architecture is sufficient to unleash these cues effectively. By aligning feature scales across encoder-decoder and enabling streamlined cross-scale propagation, our symmetric design preserves intrinsic degradation signals robustly, rendering simple additive fusion in skip connections sufficient for state-of-the-art performance. Our primary baseline, SymUNet, is built on this symmetric U-Net and achieves better results across benchmark datasets than existing approaches while reducing computational cost. We further propose a semantic enhanced variant, SE-SymUNet, which integrates direct semantic injection from frozen CLIP features via simple cross-attention to explicitly amplify degradation priors. Extensive experiments on several benchmarks validate the superiority of our methods. Both baselines SymUNet and SE-SymUNet establish simpler and stronger foundations for future advancements in all-in-one image restoration. The source code is available at https://github.com/WenlongJiao/SymUNet.

Problem

• Unified framework for diverse image degradations
• Simplify complex architectures for image restoration
• Enhance degradation cues with symmetric U-Net design

Innovation

• Symmetric U-Net architecture simplifies feature extraction
• Cross-scale propagation preserves degradation signals robustly
• CLIP semantic injection enhances degradation priors via cross-attention