HiReFF: High-Resolution Feedforward Human Reconstruction from Uncalibrated Sparse-View Video

2026-06-28

AI Summary

This work addresses the challenge of achieving high-resolution, temporally coherent, and computationally efficient human reconstruction from uncalibrated sparse-view videos. The authors propose a feed-forward two-stage framework: first reconstructing a foreground 3D Gaussian representation from sparse views, then efficiently synthesizing high-resolution images. Key innovations include scale-synchronized camera calibration to resolve multi-view scale ambiguity, Gaussian-level foreground masking for clean foreground separation, and a high-resolution side-tuning architecture that enables 2K 360° dynamic human reconstruction with low computational overhead. Experiments demonstrate that the method significantly outperforms existing approaches using only four uncalibrated input views spaced at 90° intervals, showing strong potential for applications in holographic communication and AR/VR.

Abstract

Uncalibrated volumetric video streaming for human reconstruction is essential for holographic communication and AR/VR, yet remains challenging due to the need for temporal consistency and computational efficiency from sparse-view inputs. Existing methods rely on per-scene optimization or calibrated cameras, while recent feed-forward models are limited to low-resolution (0.5K) single-frame synthesis. We present HiReFF, a feed-forward method for 2K-resolution 360° human video reconstruction from uncalibrated sparse-view videos. Our framework decomposes the problem into two key tasks: foreground 3D Gaussian reconstruction from sparse-view videos (four views separated by 90°) and computationally efficient high-resolution synthesis. To enable the former, we propose Scale-synchronized Camera Calibration to resolve scale ambiguity for multi-view supervision, and Gaussian-wise Foreground Masking to reconstruct clean foregrounds by modulating Gaussian parameters. For efficient high-resolution synthesis, our High-resolution Side-tuning achieves 2K rendering by augmenting the Gaussian head with supplementary features while keeping the backbone at 0.5K, drastically reducing computational overhead. Experiments demonstrate that HiReFF significantly outperforms existing methods in high-resolution streaming volumetric video reconstruction. https://iridescentjiang.github.io/HiReFF

Problem

• human reconstruction
• uncalibrated sparse-view video
• high-resolution
• volumetric video
• temporal consistency

Innovation

• feed-forward reconstruction
• uncalibrated sparse-view video
• 3D Gaussian splatting
• high-resolution synthesis
• scale-synchronized calibration