FaceSnap: Real-Time Personalized Lightstage Facial Performance Capture

2026-08-31

AI Summary

FaceSnap通过两阶段方法，使用单个相机实现实时高精度面部捕捉，解决传统光舞台面部捕捉资源密集问题。

Abstract

Lightstage facial capture produces production-quality digital humans, but it is resource and labor-intensive. Multi-camera setups, hours of computation, and massive data storage create bottlenecks that hinder iterative workflows. This paper introduces FaceSnap, an end-to-end framework that streamlines capture via a two-stage approach. First, a one-time multi-view optimization from a range-of-motion sequence builds a personalized model encoding both geometry and expression-dependent appearance. This model then enables high-fidelity real-time facial performance capture from a single monocular lightstage camera, with no further multi-view capture required. FaceSnap jointly estimates geometry and dynamic 4K texture at 83 fps. The 4K texture is produced by a novel personalized residual upscaler that recovers subject-specific high-frequency detail, which generic upscalers fail to capture. FaceSnap achieves geometric accuracy competitive with full per-frame multi-view optimization while outperforming feed-forward methods trained on production-quality 3D data, all from a single camera view. Finally, we introduce Multi4D, a public benchmark for evaluating 4D facial reconstruction methods in lightstage environments, enabling topology-invariant geometric comparison across methods.

Problem

• Lightstage
• facial capture
• resource-intensive
• multi-camera setups
• data storage

Innovation

• end-to-end framework
• personalized model
• real-time facial performance capture
• residual upscaler
• Multi4D benchmark