Multi-Cali Anything: Dense Feature Multi-Frame Structure-from-Motion for Large-Scale Camera Array Calibration

2025-03-02

AI Summary

To address the inefficiency and reliance on specialized calibration targets in calibrating large, fixed camera arrays (e.g., dome systems), this paper proposes a novel calibration method that requires no dedicated calibration imagery and instead jointly optimizes intrinsic and extrinsic parameters from multiple natural-scene frames. Our approach extends the Structure-from-Motion (SfM) framework with three key innovations: (i) geometric regularization of extrinsic parameters to enforce physical rigidity among fixed cameras; (ii) a dense feature reprojection loss to enhance correspondence robustness; and (iii) variance constraints on intrinsic parameters to suppress inter-camera parameter drift. The method integrates dense matching, multi-frame joint nonlinear optimization, and geometric consistency modeling. Evaluated on the Multiface dataset, it reduces intrinsic parameter estimation error by 42% and improves 3D reconstruction accuracy by 31%, achieving performance on par with conventional calibration pipelines while remaining fully compatible with standard SfM workflows.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Calibrating large-scale camera arrays, such as those in dome-based setups, is time-intensive and typically requires dedicated captures of known patterns. While extrinsics in such arrays are fixed due to the physical setup, intrinsics often vary across sessions due to factors like lens adjustments or temperature changes. In this paper, we propose a dense-feature-driven multi-frame calibration method that refines intrinsics directly from scene data, eliminating the necessity for additional calibration captures. Our approach enhances traditional Structure-from-Motion (SfM) pipelines by introducing an extrinsics regularization term to progressively align estimated extrinsics with ground-truth values, a dense feature reprojection term to reduce keypoint errors by minimizing reprojection loss in the feature space, and an intrinsics variance term for joint optimization across multiple frames. Experiments on the Multiface dataset show that our method achieves nearly the same precision as dedicated calibration processes, and significantly enhances intrinsics and 3D reconstruction accuracy. Fully compatible with existing SfM pipelines, our method provides an efficient and practical plug-and-play solution for large-scale camera setups. Our code is publicly available at: https://github.com/YJJfish/Multi-Cali-Anything

Problem

• Calibrates large-scale camera arrays without additional captures
• Refines intrinsics using scene data and dense features
• Improves 3D reconstruction accuracy and intrinsics precision

Innovation

• Dense-feature-driven multi-frame calibration method
• Extrinsics regularization for alignment with ground-truth
• Joint optimization of intrinsics across multiple frames