GUSH3R: Everyone Everywhere All at Once as Gaussians

2026-07-06

AI Summary

This work addresses the challenge of jointly reconstructing dynamic humans and static scenes from monocular video by proposing a feed-forward 3D Gaussian splatting representation. It introduces, for the first time, a unified differentiable model that represents both non-rigidly moving humans and static environments as a set of 3D Gaussians. An end-to-end network directly predicts Gaussian parameters from input video frames, enabling geometrically consistent, online reconstruction with novel view synthesis capability in a single forward pass. In contrast to existing optimization-based approaches, the proposed framework achieves significantly improved inference efficiency while maintaining high rendering quality, demonstrating superior performance in novel view synthesis on human-scene datasets.

Abstract

Reconstructing dynamic human-scene environments from monocular videos is a challenging problem that requires jointly modeling scene geometry, camera motion, and non-rigid human dynamics while enabling photorealistic rendering. Recent feed-forward methods can efficiently predict geometry, but they are often limited to non-photorealistic representations such as point clouds and meshes, or they fail to handle non-rigid objects, particularly dynamic humans. To fill this gap, we present GUSH3R (Gaussian-Unified Scene Human 3D Reconstruction), a feed-forward framework for online dynamic human-scene reconstruction. From a monocular human-scene video, our method reconstructs dynamic humans (everyone) and static scenes (everywhere) in a single forward pass (all at once) as 3D Gaussian Splatting (3DGS) primitives (as gaussians), which are geometrically consistent and capable of novel view synthesis. Experiments on monocular human-scene datasets demonstrate that our approach achieves competitive novel view synthesis quality while significantly improving inference efficiency compared to optimization-based methods.

Problem

• dynamic human-scene reconstruction
• monocular video
• non-rigid human dynamics
• photorealistic rendering
• novel view synthesis

Innovation

• 3D Gaussian Splatting
• dynamic human reconstruction
• monocular video
• feed-forward framework
• novel view synthesis