KPLM-STA: Physically-Accurate Shadow Synthesis for Human Relighting via Keypoint-Based Light Modeling

2025-11-11

AI Summary

This work addresses the longstanding challenge in portrait relighting—balancing visual plausibility and geometric fidelity in shadow synthesis. We propose a keypoint-based, physics-driven shadow modeling framework. Methodologically, we innovatively integrate a nine-keypoint linear pose model (KPLM) with a shadow triangulation algorithm (STA) to achieve geometrically accurate, joint-level dynamic shadow projection; we further couple this geometric backbone with a diffusion model to synthesize optically consistent, high-fidelity shadows. Evaluated under complex poses and multi-directional relighting conditions, our approach significantly improves spatial coherence and physical plausibility of shadows. Quantitative and perceptual assessments across multiple realism metrics demonstrate state-of-the-art performance. Moreover, the framework exhibits strong generalization capability while preserving precise anatomical geometry—enabling robust, physically grounded shadow generation for diverse relighting scenarios.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Reasoning under Uncertainty: Relational Probabilistic Models
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Image composition aims to seamlessly integrate a foreground object into a background, where generating realistic and geometrically accurate shadows remains a persistent challenge. While recent diffusion-based methods have outperformed GAN-based approaches, existing techniques, such as the diffusion-based relighting framework IC-Light, still fall short in producing shadows with both high appearance realism and geometric precision, especially in composite images. To address these limitations, we propose a novel shadow generation framework based on a Keypoints Linear Model (KPLM) and a Shadow Triangle Algorithm (STA). KPLM models articulated human bodies using nine keypoints and one bounding block, enabling physically plausible shadow projection and dynamic shading across joints, thereby enhancing visual realism. STA further improves geometric accuracy by computing shadow angles, lengths, and spatial positions through explicit geometric formulations. Extensive experiments demonstrate that our method achieves state-of-the-art performance on shadow realism benchmarks, particularly under complex human poses, and generalizes effectively to multi-directional relighting scenarios such as those supported by IC-Light.

Problem

• Generating realistic shadows with geometric accuracy in image composition
• Improving shadow appearance realism for articulated human bodies under complex poses
• Enhancing shadow projection precision in multi-directional relighting scenarios

Innovation

• Keypoints Linear Model for human shadow projection
• Shadow Triangle Algorithm for geometric accuracy
• Physically accurate multi-directional relighting framework