Uncertainty-Aware 4D Gaussian Splatting for Monocular Occluded Human Rendering

2026-02-06

AI Summary

Rendering dynamic humans from monocular videos often suffers from artifacts and temporal flickering in occluded regions due to unreliable observations. This work proposes an uncertainty-aware 4D Gaussian splatting framework that formulates rendering as a maximum a posteriori estimation problem under heteroscedastic noise. A probabilistic deformation network coupled with a dual-rasterization pipeline generates pixel-wise uncertainty maps, which are leveraged to adaptively modulate gradients during optimization to suppress artifacts. Furthermore, a confidence-aware regularization term is introduced to mitigate geometric drift. To our knowledge, this is the first approach to exploit pixel-level uncertainty for both gradient modulation and spatiotemporal regularization in dynamic human rendering. Extensive experiments on the ZJU-MoCap and OcMotion datasets demonstrate significant improvements in rendering fidelity and robustness, effectively alleviating occlusion-induced distortions and temporal inconsistencies.

Technology Category

• Computer Vision: Motion & Tracking
• Reasoning under Uncertainty: Uncertainty Representations
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

High-fidelity rendering of dynamic humans from monocular videos typically degrades catastrophically under occlusions. Existing solutions incorporate external priors-either hallucinating missing content via generative models, which induces severe temporal flickering, or imposing rigid geometric heuristics that fail to capture diverse appearances. To this end, we reformulate the task as a Maximum A Posteriori estimation problem under heteroscedastic observation noise. In this paper, we propose U-4DGS, a framework integrating a Probabilistic Deformation Network and a Double Rasterization pipeline. This architecture renders pixel-aligned uncertainty maps that act as an adaptive gradient modulator, automatically attenuating artifacts from unreliable observations. Furthermore, to prevent geometric drift in regions lacking reliable visual cues, we enforce Confidence-Aware Regularizations, which leverage the learned uncertainty to selectively propagate spatial-temporal validity. Extensive experiments on ZJU-MoCap and OcMotion demonstrate that U-4DGS achieves SOTA rendering fidelity and robustness.

Problem

• occluded human rendering
• monocular video
• 4D reconstruction
• uncertainty modeling
• dynamic human

Innovation

• Uncertainty-Aware Rendering
• 4D Gaussian Splatting
• Probabilistic Deformation Network
• Heteroscedastic Noise Modeling
• Confidence-Aware Regularization