SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras

2026-03-27

AI Summary

High-fidelity 4D dynamic scene reconstruction typically relies on dense, synchronized, and expensive multi-camera systems, which are difficult to scale. This work proposes an end-to-end reconstruction framework that operates with sparse or even uncalibrated camera inputs. The key innovation is the introduction of a Spatio-Temporal Distortion Field, which jointly models inconsistencies in generative observations across both spatial and temporal dimensions. By integrating generative observations, the spatio-temporal distortion field, and a neural radiance field, the method effectively disentangles observation noise from true scene dynamics, substantially reducing dependence on precise camera configurations. Experiments demonstrate that the approach achieves high-fidelity, temporally coherent rendering on multi-view dynamic scene benchmarks, significantly outperforming existing methods.

Abstract

High-quality 4D reconstruction enables photorealistic and immersive rendering of the dynamic real world. However, unlike static scenes that can be fully captured with a single camera, high-quality dynamic scenes typically require dense arrays of tens or even hundreds of synchronized cameras. Dependence on such costly lab setups severely limits practical scalability. The reliance on such costly lab setups severely limits practical scalability. To this end, we propose a sparse-camera dynamic reconstruction framework that exploits abundant yet inconsistent generative observations. Our key innovation is the Spatio-Temporal Distortion Field, which provides a unified mechanism for modeling inconsistencies in generative observations across both spatial and temporal dimensions. Building on this, we develop a complete pipeline that enables 4D reconstruction from sparse and uncalibrated camera inputs. We evaluate our method on multi-camera dynamic scene benchmarks, achieving spatio-temporally consistent high-fidelity renderings and significantly outperforming existing approaches.

Problem

• 4D reconstruction
• sparse cameras
• dynamic scenes
• spatio-temporal consistency
• uncalibrated cameras

Innovation

• Spatio-Temporal Distortion Field
• 4D reconstruction
• sparse cameras
• dynamic scenes
• uncalibrated inputs