Efficient Deep Demosaicing with Spatially Downsampled Isotropic Networks

2026-01-02

arXiv.org

AI Summary

This work addresses the challenge of efficiently performing image demosaicing on mobile devices by reconciling computational efficiency with high reconstruction quality. Departing from the conventional design paradigm of isotropic networks that avoid spatial downsampling, the authors propose incorporating moderate spatial downsampling into a fully convolutional isotropic architecture based on DeepMAD. The resulting model, termed JD3Net, demonstrates significant performance gains over non-downsampling baselines across multiple benchmark tasks for both demosaicing and joint demosaicing-denoising. Notably, JD3Net achieves substantially reduced computational costs while maintaining superior reconstruction fidelity, thereby validating the effectiveness and practicality of strategic spatial downsampling within isotropic network designs.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

In digital imaging, image demosaicing is a crucial first step which recovers the RGB information from a color filter array (CFA). Oftentimes, deep learning is utilized to perform image demosaicing. Given that most modern digital imaging applications occur on mobile platforms, applying deep learning to demosaicing requires lightweight and efficient networks. Isotropic networks, also known as residual-in-residual networks, have been often employed for image demosaicing and joint-demosaicing-and-denoising (JDD). Most demosaicing isotropic networks avoid spatial downsampling entirely, and thus are often prohibitively expensive computationally for mobile applications. Contrary to previous isotropic network designs, this paper claims that spatial downsampling to a signficant degree can improve the efficiency and performance of isotropic networks. To validate this claim, we design simple fully convolutional networks with and without downsampling using a mathematical architecture design technique adapted from DeepMAD, and find that downsampling improves empirical performance. Additionally, empirical testing of the downsampled variant, JD3Net, of our fully convolutional networks reveals strong empirical performance on a variety of image demosaicing and JDD tasks.

Problem

• image demosaicing
• efficient deep learning
• isotropic networks
• spatial downsampling
• mobile imaging

Innovation

• spatial downsampling
• isotropic networks
• image demosaicing
• efficient deep learning
• joint-demosaicing-and-denoising