Hand Shadow Art: A Differentiable Rendering Perspective

2025-05-27

AI Summary

This work addresses the problem of computational hand shadow art generation. We propose the first differentiable rendering-based method for 3D hand deformation inversion: given a target 2D shadow image and illumination conditions, it jointly optimizes the geometry and pose of both hands along with lighting parameters to minimize the discrepancy between rendered and target shadows. Our approach integrates neural implicit hand representations, physically grounded shadow modeling, and a gradient-guided co-optimization framework—enabling simultaneous bilateral hand solving and smooth pose interpolation across semantically distinct shadows. Experiments demonstrate stable reconstruction of high-fidelity hand shadows, precise matching of intricate shadow structures, and seamless temporal transitions. This work establishes a novel paradigm and practical toolkit for applying differentiable graphics to digital artistic creation.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: Manipulation
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Shadow art is an exciting form of sculptural art that produces captivating artistic effects through the 2D shadows cast by 3D shapes. Hand shadows, also known as shadow puppetry or shadowgraphy, involve creating various shapes and figures using your hands and fingers to cast meaningful shadows on a wall. In this work, we propose a differentiable rendering-based approach to deform hand models such that they cast a shadow consistent with a desired target image and the associated lighting configuration. We showcase the results of shadows cast by a pair of two hands and the interpolation of hand poses between two desired shadow images. We believe that this work will be a useful tool for the graphics community.

Problem

• Deform hand models for desired shadow images
• Achieve shadow consistency with lighting configuration
• Interpolate hand poses between target shadows

Innovation

• Differentiable rendering for hand shadow deformation
• Hand models cast shadows matching target images
• Interpolation of hand poses between shadow images