GVTNet: Graph Vision Transformer For Face Super-Resolution

2025-02-18

AI Summary

Existing facial super-resolution methods neglect structural correlations among facial components, leading to geometric distortions and texture artifacts in reconstructed faces. To address this, we propose GraphViT, the first framework that deeply integrates graph neural networks (GNNs) with vision transformers (ViTs). Specifically, image patches are modeled as graph nodes, and a semantic-driven adaptive adjacency matrix is learned to explicitly capture spatial dependencies among facial components; additionally, patch-level graph convolution is introduced to strengthen local structural constraints. Extensive experiments on CelebA and FFHQ demonstrate that GraphViT significantly outperforms state-of-the-art methods—including FSRNet, GPEN, and WaveLET—achieving substantial gains in PSNR and SSIM. Notably, it delivers superior structural fidelity and texture clarity in critical regions such as eyes and lips, validating its effectiveness in modeling facial topology and semantics.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Graph-based Machine Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Recent advances in face super-resolution research have utilized the Transformer architecture. This method processes the input image into a series of small patches. However, because of the strong correlation between different facial components in facial images. When it comes to super-resolution of low-resolution images, existing algorithms cannot handle the relationships between patches well, resulting in distorted facial components in the super-resolution results. To solve the problem, we propose a transformer architecture based on graph neural networks called graph vision transformer network. We treat each patch as a graph node and establish an adjacency matrix based on the information between patches. In this way, the patch only interacts between neighboring patches, further processing the relationship of facial components. Quantitative and visualization experiments have underscored the superiority of our algorithm over state-of-the-art techniques. Through detailed comparisons, we have demonstrated that our algorithm possesses more advanced super-resolution capabilities, particularly in enhancing facial components. The PyTorch code is available at https://github.com/continueyang/GVTNet

Problem

• Handles patch relationships in face super-resolution
• Prevents distortion of facial components
• Enhances facial component clarity using graph neural networks

Innovation

• Transformer architecture with graph neural networks
• Patches as graph nodes with adjacency matrix
• Enhanced super-resolution for facial components