Do Image Editing Models Understand Lighting?

2026-06-25

AI Summary

This study investigates whether current image editing models genuinely adhere to the physical laws of real-world illumination. To this end, we introduce 3DLP, the first 3D-anchored light probe benchmark dataset based on 1,000 pairs of real HDR indoor scenes captured with lights switched on and off, augmented with region-level semantic annotations. We propose a novel scoring mechanism that mitigates photographic artifacts in AI-generated images, enabling a systematic evaluation of mainstream models’ consistency with authentic light transport during illumination editing. Experiments reveal that while state-of-the-art models perform reasonably well in rendering specular highlights, they exhibit significant deviations in low-light regions. Furthermore, our analysis demonstrates that vision-language models (VLMs) are ill-suited for pixel-level illumination assessment.

Abstract

While recent advancements in generative image editing models have achieved stunning visual fidelity, it remains an open question whether these systems possess an intrinsic knowledge of real-world lighting. Existing benchmarks typically evaluate high-level plausibility of perceptual light transport on curated internet imagery, using VLMs or human judgement, or they rely on synthetically generated datasets. In this work, we introduce the 3D-anchored Light Probe (3DLP) benchmark, for which we have captured a new high-fidelity HDR dataset of real-world lighting changes. The dataset consists of 1K image pairs of diverse indoor scenery in which light probes are physically turned on and off. To allow for a granular performance analysis, we annotated specific image regions such as cast shadows or metallic surfaces. With this data, we evaluate a range of state-of-the-art image editing models by measuring how well their light probe edits align with reality. The evaluation uses two new scores to compensate for AI-generated photographic effects, such as adjusted white balance. Our results show that the overall performance of models differs considerably, with differences slightly less pronounced for specular highlights. The best image editing models are remarkably consistent with real-world physics, however, they still leave room for improvement. We observe that image regions that receive less light from the light probe are more prone to errors for all models. Furthermore, building on their success in evaluating macroscopic lighting plausibility, we test VLMs on our task but find that they are unsuitable for pixel-level light transport analysis. We will make the benchmark, together with the real-world dataset, publicly available to encourage future research on this topic.

Problem

• image editing
• lighting understanding
• light transport
• real-world physics
• generative models

Innovation

• 3D-anchored Light Probe
• HDR lighting dataset
• pixel-level light transport
• generative image editing
• lighting consistency evaluation