4D Human-Scene Reconstruction from Low-Overlap Captures

2026-07-10

AI Summary

This work addresses the challenges of 4D human reconstruction in real-world multi-view settings with low camera overlap, where existing methods often suffer from geometric artifacts due to insufficient observations and video diffusion models struggle to maintain temporal geometric consistency. To overcome these limitations, we propose StudioRecon, a novel framework that first disentangles the background from the human subject and leverages a video diffusion model to synthesize dense, controllable novel views for enhanced background supervision. We initialize a deformable Gaussian representation of the human body by integrating cross-view identity associations and multi-view keypoint triangulation. Furthermore, a motion-adaptive recursive refinement module is introduced to enforce temporal consistency. Evaluated on four real-world datasets, our method achieves state-of-the-art novel view synthesis performance, significantly reducing artifacts and enabling applications such as trajectory re-rendering and human replacement.

Abstract

Existing volumetric capture of dynamic human performance achieves high fidelity with dense camera arrays. However, in real-world scenarios, only a handful of low-overlap cameras are available, which degrades the output quality and leaves large areas unobserved. Recent 4D reconstruction methods have focused on low-overlap settings, yet they still produce noticeable artifacts in under-observed regions. Video diffusion models have emerged as another option, but they show geometrically inconsistent results for humans. To address these limitations, we propose StudioRecon, a pipeline that reconstructs 4D human scenes from sparse, low-overlap cameras by decoupling background and humans. We densify background supervision by synthesizing hundreds of camera-controlled novel views with a video diffusion model. We also robustly initialize deformable Gaussian humans with cross-view identity association and triangulated multi-view keypoint fitting. Finally, our recursive enhancement module with motion-adaptive consistency injection harmonizes the composed output, thereby further avoiding remaining artifacts. We achieve state-of-the-art novel view synthesis across four real-world datasets and demonstrate applications such as novel trajectory rendering and human replacement.

Problem

• 4D reconstruction
• low-overlap cameras
• human-scene reconstruction
• sparse views
• geometric consistency

Innovation

• 4D reconstruction
• low-overlap cameras
• video diffusion model
• deformable Gaussians
• motion-adaptive consistency