SemiISP/SemiIE: Semi-Supervised Image Signal Processor and Image Enhancement Leveraging One-to-Many Mapping sRGB-to-RAW

2025-04-03

AI Summary

To address the scarcity of high-quality RAW annotations and the difficulty in satisfying personalized requirements in ISP and image enhancement (IE), this paper proposes the first semi-supervised learning framework tailored to this domain. Methodologically, it introduces: (1) an optimized one-to-many sRGB-to-RAW inverse mapping specifically designed for ISP/IE, substantially improving the fidelity of pseudo-RAW data; (2) a conditional generative modeling scheme, an adversarial-aware loss function, and a self-distillation-based pseudo-label refinement mechanism. Empirically, the framework achieves full-supervision performance using only 10% real RAW annotations. It delivers significant PSNR and SSIM improvements across multiple benchmarks and demonstrates strong generalization capabilities across diverse camera models and user-specific preferences.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web

Abstract

DNN-based methods have been successful in Image Signal Processor (ISP) and image enhancement (IE) tasks. However, the cost of creating training data for these tasks is considerably higher than for other tasks, making it difficult to prepare large-scale datasets. Also, creating personalized ISP and IE with minimal training data can lead to new value streams since preferred image quality varies depending on the person and use case. While semi-supervised learning could be a potential solution in such cases, it has rarely been utilized for these tasks. In this paper, we realize semi-supervised learning for ISP and IE leveraging a RAW image reconstruction (sRGB-to-RAW) method. Although existing sRGB-to-RAW methods can generate pseudo-RAW image datasets that improve the accuracy of RAW-based high-level computer vision tasks such as object detection, their quality is not sufficient for ISP and IE tasks that require precise image quality definition. Therefore, we also propose a sRGB-to-RAW method that can improve the image quality of these tasks. The proposed semi-supervised learning with the proposed sRGB-to-RAW method successfully improves the image quality of various models on various datasets.

Problem

• Reducing high cost of training data creation for ISP and IE tasks
• Enabling personalized ISP and IE with minimal training data
• Improving sRGB-to-RAW quality for semi-supervised ISP and IE

Innovation

• Semi-supervised learning for ISP and IE
• Improved sRGB-to-RAW reconstruction method
• One-to-many mapping for personalized image quality