RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restoration

2025-05-23

AI Summary

To address the slow inference and poor real-time deployability of latent diffusion models (LDMs) in all-in-one image restoration (AiOR), this paper pioneers the integration of vision autoregressive (VAR) modeling into AiOR, proposing an efficient generative framework. Methodologically, it replaces iterative denoising with single-step generation. Key contributions include: (1) a cross-scale cross-attention mechanism that enhances multi-granularity feature interaction; and (2) a latent-space refinement module that improves reconstruction fidelity and generalization. Evaluated on multiple AiOR benchmarks, the method achieves state-of-the-art performance, accelerating inference by over 10× compared to LDMs while preserving strong generalization across diverse degradation types. This work significantly advances the practical deployment of generative image restoration.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The use of latent diffusion models (LDMs) such as Stable Diffusion has significantly improved the perceptual quality of All-in-One image Restoration (AiOR) methods, while also enhancing their generalization capabilities. However, these LDM-based frameworks suffer from slow inference due to their iterative denoising process, rendering them impractical for time-sensitive applications. To address this, we propose RestoreVAR, a novel generative approach for AiOR that significantly outperforms LDM-based models in restoration performance while achieving over $mathbf{10 imes}$ faster inference. RestoreVAR leverages visual autoregressive modeling (VAR), a recently introduced approach which performs scale-space autoregression for image generation. VAR achieves comparable performance to that of state-of-the-art diffusion transformers with drastically reduced computational costs. To optimally exploit these advantages of VAR for AiOR, we propose architectural modifications and improvements, including intricately designed cross-attention mechanisms and a latent-space refinement module, tailored for the AiOR task. Extensive experiments show that RestoreVAR achieves state-of-the-art performance among generative AiOR methods, while also exhibiting strong generalization capabilities.

Problem

• Improving slow inference in LDM-based All-in-One image Restoration
• Enhancing restoration performance with visual autoregressive modeling
• Reducing computational costs while maintaining high-quality image generation

Innovation

• Visual autoregressive modeling for image restoration
• Cross-attention mechanisms for enhanced performance
• Latent-space refinement module for AiOR tasks