UniCon3R: Contact-aware 3D Human-Scene Reconstruction from Monocular Video

2026-04-21

AI Summary

This work addresses the common issue of physically implausible artifacts—such as floating or interpenetration—in 4D reconstructions of humans and scenes from monocular video, which often arise due to the absence of explicit physical interaction modeling. To this end, the authors propose UniCon3R, a unified feed-forward framework that, for the first time, incorporates human-scene contact relationships as an internal corrective prior during reconstruction rather than merely as a post-hoc evaluation metric. UniCon3R jointly infers high-fidelity scene geometry and spatially aligned 3D human poses from monocular video in an online manner, while simultaneously leveraging predicted 3D contact signals to co-optimize both representations. Evaluated on multiple benchmarks—including RICH, EMDB, 3DPW, and SLOPER4D—the method significantly outperforms existing approaches in terms of physical plausibility and global human motion accuracy, while also enabling real-time inference.

Technology Category

• Computer Vision: 3D Computer Vision
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We introduce UniCon3R (Unified Contact-aware 3D Reconstruction), a unified feed-forward framework for online human-scene 4D reconstruction from monocular videos. Recent feed-forward methods enable real-time world-coordinate human motion and scene reconstruction, but they often produce physically implausible artifacts such as bodies floating above the ground or penetrating parts of the scene. The key reason is that existing approaches fail to model physical interactions between the human and the environment. A natural next step is to predict human-scene contact as an auxiliary output -- yet we find this alone is not sufficient: contact must actively correct the reconstruction. To address this, we explicitly model interaction by inferring 3D contact from the human pose and scene geometry and use the contact as a corrective cue for generating the final pose. This enables UniCon3R to jointly recover high-fidelity scene geometry and spatially aligned 3D humans within the scene. Experiments on standard human-centric video benchmarks such as RICH, EMDB, 3DPW and SLOPER4D show that UniCon3R outperforms state-of-the-art baselines on physical plausibility and global human motion estimation while achieving real-time online inference. We experimentally demonstrate that contact serves as a powerful internal prior rather than just an external metric, thus establishing a new paradigm for physically grounded joint human-scene reconstruction. Project page is available at https://surtantheta.github.io/UniCon3R .

Problem

• human-scene interaction
• physical plausibility
• 3D reconstruction
• monocular video
• contact modeling

Innovation

• contact-aware reconstruction
• monocular video
• human-scene interaction
• physical plausibility
• real-time 4D reconstruction