HIR-ALIGN: Enhancing Hyperspectral Image Restoration via Diffusion-Based Data Generation

2026-05-13

AI Summary

This work addresses the challenge of hyperspectral image restoration in real-world scenarios, where degradations such as noise, blur, and low resolution are prevalent, and clean reference images are unavailable in the target domain, leading to poor generalization of models trained solely on source-domain data. To tackle this, we propose HIR-ALIGN, a novel framework that uniquely integrates diffusion models with spectral transfer for unsupervised domain adaptation without requiring additional data. Our approach employs a three-stage pipeline: first generating semantic-preserving proxy hyperspectral images, then synthesizing RGB images aligned with the target distribution using an enhanced blur-robust unCLIP diffusion model, and finally producing hyperspectral data via warp-based spectral transfer. The restored image is obtained by fine-tuning a recovery network on a fusion of proxy and synthesized data. Theoretical analysis shows reduced restoration risk in the target domain, and experiments demonstrate that HIR-ALIGN significantly outperforms existing source-only and unsupervised methods in both denoising and super-resolution tasks on simulated and real datasets.

Abstract

Hyperspectral image (HSI) restoration is crucial for reliable analysis, as real HSIs suffer from degradations like noise, blur, and resolution loss. However, existing models trained on source data often fail on target domains lacking clean references, a common occurrence in practice. To address this issue, we present HIR-ALIGN, a plug-and-play target-adaptive augmentation framework that enhances hyperspectral image restoration by augmenting limited training images with synthetic data that closely matches the target distribution using no extra data. It consists of three stages: (i) proxy generation, where off-the-shelf restoration models restore degraded target observations to produce semantics-preserving proxy HSIs that approximate target-domain clean images; (ii) distribution-adaptive synthesis, where a blur-robust unCLIP diffusion model generates target-aligned RGBs from proxy RGBs, with prompt conditioning and embedding-space noise initialization. Then, a warp-based spectral transfer module synthesizes HSIs by aligning each generated RGB with the proxy RGB, estimating soft patch-wise transport weights, and applying these weights and learnable local interpolation kernels to the proxy HSI; and (iii) aligned supervised finetuning, where restoration networks pretrained on the source distribution are finetuned using both the proxy HSIs and synthesized target-aligned HSIs, and are then deployed on degraded target images. We further provide theoretical analysis showing that augmentation-based finetuning can achieve lower target-domain restoration risk by jointly improving target distribution coverage and controlling spectral bias. Extensive experiments on simulated and real datasets across denoising and super-resolution tasks demonstrate that HIR-ALIGN consistently improves source-only supervised baselines, outperforming both source-only counterparts and representative unsupervised methods.

Problem

• hyperspectral image restoration
• domain adaptation
• target domain
• clean reference
• degradation

Innovation

• diffusion-based data generation
• target-adaptive augmentation
• hyperspectral image restoration
• distribution alignment
• spectral transfer