Enhanced Self-Supervised Multi-Image Super-Resolution for Camera Array Images

2026-04-08

AI Summary

This work addresses the challenges of irregular spatial sampling and difficulty in recovering high-frequency details in camera array image super-resolution, where existing self-supervised methods often yield limited reconstruction quality. To overcome these limitations, the paper proposes a novel self-supervised framework that introduces a Multi-to-Single guided Multi-to-Multi learning paradigm. This approach synergistically integrates a physics-driven variational model with deep neural networks and incorporates a dual-Transformer architecture specifically designed for self-supervised learning. Extensive experiments on both synthetic and real-world camera array datasets demonstrate that the proposed method substantially outperforms current state-of-the-art techniques, achieving significant improvements in reconstruction accuracy and fidelity of fine texture details.

Abstract

Conventional multi-image super-resolution (MISR) methods, such as burst and video SR, rely on sequential frames from a single camera. Consequently, they suffer from complex image degradation and severe occlusion, increasing the difficulty of accurate image restoration. In contrast, multi-aperture camera-array imaging captures spatially distributed views with sampling offsets forming a stable disk-like distribution, which enhances the non-redundancy of observed data. Existing MISR algorithms fail to fully exploit these unique properties. Supervised MISR methods tend to overfit the degradation patterns in training data, and current self-supervised learning (SSL) techniques struggle to recover fine-grained details. To address these issues, this paper thoroughly investigates the strengths, limitations and applicability boundaries of multi-image-to-single-image (Multi-to-Single) and multi-image-to-multi-image (Multi-to-Multi) SSL methods. We propose the Multi-to-Single-Guided Multi-to-Multi SSL framework that combines the advantages of Multi-to-Single and Multi-to-Multi to generate visually appealing and high-fidelity images rich in texture details. The Multi-to-Single-Guided Multi-to-Multi SSL framework provides a new paradigm for integrating deep neural network with classical physics-based variational methods. To enhance the ability of MISR network to recover high-frequency details from aliased artifacts, this paper proposes a novel camera-array SR network called dual Transformer suitable for SSL. Experiments on synthetic and real-world datasets demonstrate the superiority of the proposed method.

Problem

• multi-image super-resolution
• camera array
• self-supervised learning
• image restoration
• high-frequency details

Innovation

• self-supervised learning
• multi-image super-resolution
• camera array
• dual Transformer
• Multi-to-Multi