Multi-level distortion-aware deformable network for omnidirectional image super-resolution

2025-12-19

AI Summary

Equi-rectangular projection (ERP) images suffer from latitude-dependent geometric distortion—minimal near the equator but severe polar stretching—posing a fundamental challenge for omnidirectional image super-resolution (ODISR). Conventional ODISR methods exhibit limited receptive fields, hindering effective modeling of large-scale, non-uniform distortions. To address this, we propose a multi-level distortion-aware deformable network. Our method introduces a novel three-branch parallel deformable feature extraction architecture, integrating deformable attention with multi-scale dilated deformable convolutions (dilation rates = 1, 2, 3). Furthermore, we design adaptive multi-level feature fusion and a low-rank sparsification mechanism to substantially enlarge the effective sampling range and enhance distortion modeling capability. Extensive experiments on mainstream omnidirectional image (ODI) benchmarks demonstrate that our approach achieves state-of-the-art performance, with significant PSNR and SSIM improvements—particularly in faithfully recovering texture and structural details in severely stretched polar regions.

Technology Category

• Computer Vision: Other Foundations of Computer Vision
• Search and Optimization: Distributed Search
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Location- and context-aware Web and WoT applications and services

Abstract

As augmented reality and virtual reality applications gain popularity, image processing for OmniDirectional Images (ODIs) has attracted increasing attention. OmniDirectional Image Super-Resolution (ODISR) is a promising technique for enhancing the visual quality of ODIs. Before performing super-resolution, ODIs are typically projected from a spherical surface onto a plane using EquiRectangular Projection (ERP). This projection introduces latitude-dependent geometric distortion in ERP images: distortion is minimal near the equator but becomes severe toward the poles, where image content is stretched across a wider area. However, existing ODISR methods have limited sampling ranges and feature extraction capabilities, which hinder their ability to capture distorted patterns over large areas. To address this issue, we propose a novel Multi-level Distortion-aware Deformable Network (MDDN) for ODISR, designed to expand the sampling range and receptive field. Specifically, the feature extractor in MDDN comprises three parallel branches: a deformable attention mechanism (serving as the dilation=1 path) and two dilated deformable convolutions with dilation rates of 2 and 3. This architecture expands the sampling range to include more distorted patterns across wider areas, generating dense and comprehensive features that effectively capture geometric distortions in ERP images. The representations extracted from these deformable feature extractors are adaptively fused in a multi-level feature fusion module. Furthermore, to reduce computational cost, a low-rank decomposition strategy is applied to dilated deformable convolutions. Extensive experiments on publicly available datasets demonstrate that MDDN outperforms state-of-the-art methods, underscoring its effectiveness and superiority in ODISR.

Problem

• Addresses geometric distortion in omnidirectional image super-resolution
• Expands sampling range to capture distorted patterns in ERP images
• Reduces computational cost with low-rank decomposition strategy

Innovation

• Deformable attention and dilated convolutions expand sampling range
• Multi-level feature fusion adaptively combines extracted representations
• Low-rank decomposition reduces computational cost of convolutions