Joint Shadow Generation and Relighting via Light-Geometry Interaction Maps

2026-02-25

AI Summary

This work addresses common artifacts in single-image shadow generation and relighting—such as floating shadows, lighting inconsistencies, and geometric implausibility—by proposing a Light-Geometry Interaction (LGI) mapping that formulates shadow generation and relighting as a coupled task for the first time. The method explicitly associates illumination direction with scene geometry through a 2.5D depth representation, enabling a unified framework for joint optimization. A physics-inspired rendering prior is introduced to constrain the generation process, enhancing physical plausibility. Additionally, the authors construct the first large-scale dataset for joint training of shadow and lighting estimation. Experiments demonstrate that the proposed approach significantly improves the realism and consistency of shadows and relit appearances on both synthetic and real images, particularly excelling in challenging scenarios involving reflections, transparency, and complex interreflections.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We propose Light-Geometry Interaction (LGI) maps, a novel representation that encodes light-aware occlusion from monocular depth. Unlike ray tracing, which requires full 3D reconstruction, LGI captures essential light-shadow interactions reliably and accurately, computed from off-the-shelf 2.5D depth map predictions. LGI explicitly ties illumination direction to geometry, providing a physics-inspired prior that constrains generative models. Without such prior, these models often produce floating shadows, inconsistent illumination, and implausible shadow geometry. Building on this representation, we propose a unified pipeline for joint shadow generation and relighting - unlike prior methods that treat them as disjoint tasks - capturing the intrinsic coupling of illumination and shadowing essential for modeling indirect effects. By embedding LGI into a bridge-matching generative backbone, we reduce ambiguity and enforce physically consistent light-shadow reasoning. To enable effective training, we curated the first large-scale benchmark dataset for joint shadow and relighting, covering reflections, transparency, and complex interreflections. Experiments show significant gains in realism and consistency across synthetic and real images. LGI thus bridges geometry-inspired rendering with generative modeling, enabling efficient, physically consistent shadow generation and relighting.

Problem

• shadow generation
• relighting
• light-geometry interaction
• physically consistent rendering
• monocular depth

Innovation

• Light-Geometry Interaction
• shadow generation
• relighting
• generative modeling
• physically consistent rendering