A Systematic Investigation on Deep Learning-Based Omnidirectional Image and Video Super-Resolution

2025-06-07

AI Summary

Omnidirectional image/video super-resolution (SR) is critical for VR/AR applications, yet existing methods suffer from overfitting to synthetic degradation datasets and lack rigorous generalization evaluation under real-world conditions. To address this, we introduce 360Insta—the first large-scale panoramic SR dataset featuring diverse real-world degradations, including multi-illumination, motion blur, and exposure variations. We establish a unified benchmarking framework to systematically evaluate generalization performance, exposing critical bottlenecks of state-of-the-art methods on authentic data. Our analysis identifies three pivotal research directions: projection invariance, spherical convolution, and spatiotemporal consistency. We conduct comprehensive benchmarking across 12 representative methods—including CNN- and Transformer-based architectures, ERP and CubeMap projections, multi-scale losses, and cross-dataset transfer learning. All data, code, and evaluation tools are publicly released. Under realistic degradations, our best-performing method achieves absolute improvements of +12.7 dB PSNR and +9.3% SSIM over prior baselines.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Omnidirectional image and video super-resolution is a crucial research topic in low-level vision, playing an essential role in virtual reality and augmented reality applications. Its goal is to reconstruct high-resolution images or video frames from low-resolution inputs, thereby enhancing detail preservation and enabling more accurate scene analysis and interpretation. In recent years, numerous innovative and effective approaches have been proposed, predominantly based on deep learning techniques, involving diverse network architectures, loss functions, projection strategies, and training datasets. This paper presents a systematic review of recent progress in omnidirectional image and video super-resolution, focusing on deep learning-based methods. Given that existing datasets predominantly rely on synthetic degradation and fall short in capturing real-world distortions, we introduce a new dataset, 360Insta, that comprises authentically degraded omnidirectional images and videos collected under diverse conditions, including varying lighting, motion, and exposure settings. This dataset addresses a critical gap in current omnidirectional benchmarks and enables more robust evaluation of the generalization capabilities of omnidirectional super-resolution methods. We conduct comprehensive qualitative and quantitative evaluations of existing methods on both public datasets and our proposed dataset. Furthermore, we provide a systematic overview of the current status of research and discuss promising directions for future exploration. All datasets, methods, and evaluation metrics introduced in this work are publicly available and will be regularly updated. Project page: https://github.com/nqian1/Survey-on-ODISR-and-ODVSR.

Problem

• Reconstruct high-resolution omnidirectional images and videos from low-resolution inputs
• Address lack of real-world distorted datasets for robust evaluation
• Systematically review deep learning methods for omnidirectional super-resolution

Innovation

• Deep learning-based omnidirectional super-resolution methods
• New dataset 360Insta for real-world distortions
• Comprehensive evaluation on public and new datasets