InFlux++: Real and Synthetic Data for Estimating Dynamic Camera Intrinsics

2026-07-06

AI Summary

This work addresses the limitations of existing 3D vision methods, which typically assume fixed camera intrinsics and struggle with dynamic changes—such as varying focal lengths—in real-world scenarios, further hindered by a lack of diverse training data and evaluation benchmarks. To overcome these challenges, we introduce InFlux++, comprising a large-scale synthetic dataset, InFlux++ Synth, and an expanded real-world benchmark, InFlux++ Real. The synthetic dataset is procedurally generated, providing per-frame ground-truth annotations for intrinsics, poses, depth, and surface normals, while encompassing zooming, focusing, dynamic objects, and photorealistic rendering. The real-world benchmark significantly enhances scene and motion diversity. Experiments demonstrate that models fine-tuned on InFlux++ Synth achieve substantially higher focal length estimation accuracy on real data than current state-of-the-art methods, validating the efficacy of synthetic supervision for dynamic intrinsic parameter estimation.

Abstract

Camera intrinsics are vital for recovering 3D structure from 2D video. However, most 3D algorithms assume fixed intrinsics throughout a video, an assumption that often fails for real-world in-the-wild videos. Consequently, estimating per-frame intrinsics from RGB images is critical for making 3D methods robust to videos with dynamic intrinsics. InFlux previously advanced this research direction by establishing the first real-world benchmark with per-frame ground truth intrinsics for dynamic intrinsics videos. Nevertheless, existing methods remain inaccurate due to two obstacles: (i) training data is scarce and lacks intrinsics diversity; and (ii) benchmarks, including InFlux, have limited scene and camera motion diversity, making it difficult to properly evaluate methods. To address both gaps, we present InFlux++, consisting of two components. InFlux++ Synth is a large-scale procedurally generated synthetic video dataset with 441K+ annotated frames from 1841 high-resolution videos, providing accurate per-frame ground truth intrinsics for training dynamic intrinsics prediction models; a subset also includes per-frame pose, depth, and normals. The videos feature rich intrinsics diversity through changes in camera zoom and focus, as well as dynamic objects and realistic rendering effects such as lens distortion and defocus blur. InFlux++ Real is a large-scale real-world benchmark that extends InFlux with 514K+ newly captured frames across 334 high-resolution videos, spanning a wider range of scenes and camera motions. Finetuning existing intrinsics prediction methods on InFlux++ Synth consistently improves focal length estimation across both InFlux++ Real and InFlux, suggesting that synthetic supervision is promising for RGB-based intrinsics prediction. For the dataset, benchmark, code, videos, submission instructions, and live leaderboard, please visit https://influx.cs.princeton.edu/ .

Problem

• dynamic camera intrinsics
• 3D reconstruction
• RGB-based intrinsics estimation
• in-the-wild videos
• camera calibration

Innovation

• dynamic camera intrinsics
• synthetic dataset
• real-world benchmark
• procedural generation
• RGB-based intrinsic estimation