RefSTAR: Blind Facial Image Restoration with Reference Selection, Transfer, and Reconstruction

2025-07-14

AI Summary

Blind face image restoration faces dual challenges: unknown, complex degradations and severe identity feature loss. To address these, we propose RefSel—a novel framework that enhances restoration quality and identity fidelity through three key innovations: (1) a reference image selection module (RefSel) that constructs a high-quality RefSel-HQ dataset via mask-guided generation; (2) a cross-attention-driven feature transfer mechanism coupled with a mask-guided cycle-consistency loss to precisely propagate critical facial structures and textures; and (3) a reference image reconstruction paradigm enabling robust, identity-sensitive feature fusion in discriminative regions. Extensive experiments across multiple backbone architectures demonstrate that RefSel consistently outperforms state-of-the-art methods in PSNR, LPIPS, and identity similarity metrics. The code, RefSel-HQ dataset, and pretrained models are publicly available.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Adversarial Learning & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Security and Privacy: Large-scale security measurements

Abstract

Blind facial image restoration is highly challenging due to unknown complex degradations and the sensitivity of humans to faces. Although existing methods introduce auxiliary information from generative priors or high-quality reference images, they still struggle with identity preservation problems, mainly due to improper feature introduction on detailed textures. In this paper, we focus on effectively incorporating appropriate features from high-quality reference images, presenting a novel blind facial image restoration method that considers reference selection, transfer, and reconstruction (RefSTAR). In terms of selection, we construct a reference selection (RefSel) module. For training the RefSel module, we construct a RefSel-HQ dataset through a mask generation pipeline, which contains annotating masks for 10,000 ground truth-reference pairs. As for the transfer, due to the trivial solution in vanilla cross-attention operations, a feature fusion paradigm is designed to force the features from the reference to be integrated. Finally, we propose a reference image reconstruction mechanism that further ensures the presence of reference image features in the output image. The cycle consistency loss is also redesigned in conjunction with the mask. Extensive experiments on various backbone models demonstrate superior performance, showing better identity preservation ability and reference feature transfer quality. Source code, dataset, and pre-trained models are available at https://github.com/yinzhicun/RefSTAR.

Problem

• Blind facial image restoration with unknown degradations
• Identity preservation in facial image restoration
• Effective feature transfer from high-quality references

Innovation

• Constructs RefSel module for reference selection
• Designs feature fusion paradigm for transfer
• Proposes reference image reconstruction mechanism