Time-Aware Auto White Balance in Mobile Photography

2025-04-08

AI Summary

To address inaccurate illuminant estimation in mobile auto-white-balance (AWB) caused by sole reliance on RAW color statistics, this work pioneers the systematic integration of contextual metadata—specifically capture time and geographic location—into AWB. We propose a lightweight multimodal illuminant estimation model (≈5K parameters) featuring a novel feature interaction mechanism that jointly encodes temporal, spatial, and RAW chromatic cues. To support this approach, we introduce the first smartphone AWB benchmark dataset with user preference annotations and rich metadata (3,224 images), where ground-truth illuminants are calibrated via color charts and validated through user studies. Experiments demonstrate that our model achieves accuracy on par with or exceeding larger, state-of-the-art models, while significantly improving perceptual naturalness and user satisfaction in real-world mobile photography scenarios.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: State Estimation
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Cameras rely on auto white balance (AWB) to correct undesirable color casts caused by scene illumination and the camera's spectral sensitivity. This is typically achieved using an illuminant estimator that determines the global color cast solely from the color information in the camera's raw sensor image. Mobile devices provide valuable additional metadata-such as capture timestamp and geolocation-that offers strong contextual clues to help narrow down the possible illumination solutions. This paper proposes a lightweight illuminant estimation method that incorporates such contextual metadata, along with additional capture information and image colors, into a compact model (~5K parameters), achieving promising results, matching or surpassing larger models. To validate our method, we introduce a dataset of 3,224 smartphone images with contextual metadata collected at various times of day and under diverse lighting conditions. The dataset includes ground-truth illuminant colors, determined using a color chart, and user-preferred illuminants validated through a user study, providing a comprehensive benchmark for AWB evaluation.

Problem

• Corrects color casts in mobile photos using metadata
• Improves auto white balance with lightweight model
• Validates method with diverse lighting condition dataset

Innovation

• Uses contextual metadata for illuminant estimation
• Compact model with only ~5K parameters
• Includes user-preferred illuminants in dataset