CoGS: Compositional Dynamic Human-Object Scenes Gaussian Splatting from Monocular Video

2026-06-27

AI Summary

Reconstructing dynamic human-object interaction scenes from monocular video is highly challenging due to the distinct motion patterns of humans, objects, and background that share the same pixel space, often leading to motion ambiguity and under-constrained human geometry in sparsely observed regions. This work proposes a compositional Gaussian splatting framework that disentangles the scene into three cooperative branches—human, rigid objects, and static background—and employs a six-stage optimization strategy to sequentially stabilize each component. The approach introduces several key innovations: a complete canonical prior–based human representation, a trajectory-driven object field, and a weakly supervised planar primitive regularization for the static background. Evaluated on the HOSNeRF and NeuMan datasets, the method achieves significant improvements in reconstruction fidelity and rendering quality, outperforming existing approaches in both full-frame and human-centric metrics.

Abstract

Reconstructing dynamic human--object interaction scenes from monocular video is difficult because the human, manipulated object, and background obey different motion models while sharing the same pixels. Existing dynamic radiance-field and Gaussian-splatting methods often entangle these components, causing object motion to leak into the human or static scene, and monocular human reconstruction remains underconstrained in regions that are rarely observed. We present CoGS, a compositional Gaussian-splatting framework for monocular human--object scene reconstruction. CoGS decomposes the video into three coordinated branches: an articulated human initialized from a complete canonical prior, a rigid object field driven by an estimated object trajectory, and a static scene field regularized by weak scene-only planar primitives when available. A six-stage optimization schedule first stabilizes the human and object independently, then fuses them with the scene under full-image supervision, visibility-aware human anchoring, object silhouette and motion constraints, and delayed scene regularization. This design keeps each component responsible for its own geometry and motion while allowing photometric evidence to correct the final composite. Experiments on HOSNeRF and NeuMan show that CoGS improves both human--object interaction reconstruction and in-the-wild human--scene rendering, achieving stronger fidelity and perceptual quality across full-frame and human-focused evaluations. Code will be released upon publication.

Problem

• dynamic human-object interaction
• monocular video reconstruction
• motion disentanglement
• 3D scene decomposition
• underconstrained reconstruction

Innovation

• Compositional Gaussian Splatting
• Monocular Video Reconstruction
• Human-Object Interaction
• Dynamic Scene Modeling
• Articulated Human Prior