Automatic Image Colorization with Convolutional Neural Networks and Generative Adversarial Networks

2025-08-07

AI Summary

Grayscale image colorization is a highly ill-posed problem due to the absence of two-dimensional chrominance information, leading to multimodal color mapping ambiguities. This paper proposes a semantic-aware classification-based colorization framework that integrates convolutional neural networks (CNNs) with generative adversarial networks (GANs), formulating colorization as a classification task over a discretized color space. We jointly optimize a semantic-guided classification loss and an adversarial loss. Crucially, semantic segmentation priors are incorporated to constrain color distributions and mitigate color ambiguity, while adversarial training enhances texture fidelity and perceptual realism. Evaluated on standard benchmarks—including ImageNet and Places2—our method significantly outperforms regression-based colorization approaches, achieving state-of-the-art performance in color accuracy, semantic plausibility, and output diversity.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Image colorization, the task of adding colors to grayscale images, has been the focus of significant research efforts in computer vision in recent years for its various application areas such as color restoration and automatic animation colorization [15, 1]. The colorization problem is challenging as it is highly ill-posed with two out of three image dimensions lost, resulting in large degrees of freedom. However, semantics of the scene as well as the surface texture could provide important cues for colors: the sky is typically blue, the clouds are typically white and the grass is typically green, and there are huge amounts of training data available for learning such priors since any colored image could serve as a training data point [20]. Colorization is initially formulated as a regression task[5], which ignores the multi-modal nature of color prediction. In this project, we explore automatic image colorization via classification and adversarial learning. We will build our models on prior works, apply modifications for our specific scenario and make comparisons.

Problem

• Automatically colorize grayscale images using deep learning
• Address ill-posed nature of image colorization problem
• Explore multi-modal color prediction via adversarial learning

Innovation

• Uses Convolutional Neural Networks for color prediction
• Employs Generative Adversarial Networks for realistic results
• Formulates colorization as classification and adversarial learning