PALM: A Dataset and Baseline for Learning Multi-subject Hand Prior

2025-11-07

AI Summary

Generating high-fidelity, personalized 3D hand avatars from a single image remains challenging due to complex hand geometry, appearance variability, unconstrained lighting, and severe viewpoint ambiguity—exacerbated by the lack of large-scale, high-accuracy 3D hand data with multi-view imagery and diverse subject coverage. To address this, we introduce PALM: the first large-scale, multi-subject hand dataset comprising 263 subjects, 13K high-fidelity 3D scans, and 90K multi-view images. We further propose PALM-Net, a physics-aware joint geometric–material prior model that integrates inverse rendering with deep learning to enable single-image-driven, relightable, photorealistic hand reconstruction. PALM-Net jointly estimates geometry, albedo, specular reflectance, and normal maps while enforcing physical consistency via differentiable rendering. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches in geometric accuracy (e.g., lower Chamfer distance), material fidelity (e.g., improved SSIM on relit renderings), and subject-specific personalization.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The ability to grasp objects, signal with gestures, and share emotion through touch all stem from the unique capabilities of human hands. Yet creating high-quality personalized hand avatars from images remains challenging due to complex geometry, appearance, and articulation, particularly under unconstrained lighting and limited views. Progress has also been limited by the lack of datasets that jointly provide accurate 3D geometry, high-resolution multiview imagery, and a diverse population of subjects. To address this, we present PALM, a large-scale dataset comprising 13k high-quality hand scans from 263 subjects and 90k multi-view images, capturing rich variation in skin tone, age, and geometry. To show its utility, we present a baseline PALM-Net, a multi-subject prior over hand geometry and material properties learned via physically based inverse rendering, enabling realistic, relightable single-image hand avatar personalization. PALM's scale and diversity make it a valuable real-world resource for hand modeling and related research.

Problem

• Creating high-quality personalized hand avatars from images remains challenging
• Lack of datasets providing accurate 3D geometry and diverse subjects
• Difficulty modeling complex hand geometry under unconstrained lighting conditions

Innovation

• Large-scale multi-subject hand dataset with scans and images
• Multi-subject prior learned via inverse rendering technique
• Enables realistic relightable avatar personalization from single image