Deep Learning for Camera Calibration and Beyond: A Survey

2023-03-19

arXiv.org

AI Summary

Conventional camera calibration relies heavily on manual, labor-intensive procedures and physical calibration targets, limiting scalability and practicality. Method: This paper presents a systematic survey of deep learning–based camera calibration methods from 2014 to 2024, introducing the first unified taxonomy encompassing pinhole modeling, distortion estimation, cross-view alignment, and cross-sensor transfer. To address poor generalizability and inconsistent evaluation in existing approaches, we construct the first open-source, multimodal benchmark—integrating synthetic and real-world data—with diverse camera configurations, illumination conditions, and lens distortions. The benchmark incorporates geometric-prior-guided architectures, a cross-domain generalization evaluation protocol, and a scalable synthetic data generation pipeline. Contribution/Results: Our work yields a structured methodological taxonomy and an actively maintained resource repository, Awesome-Deep-Camera-Calibration, significantly advancing standardization, reproducibility, and empirical rigor in deep camera calibration research.

Abstract

Camera calibration involves estimating camera parameters to infer geometric features from captured sequences, which is crucial for computer vision and robotics. However, conventional calibration is laborious and requires dedicated collection. Recent efforts show that learning-based solutions have the potential to be used in place of the repeatability works of manual calibrations. Among these solutions, various learning strategies, networks, geometric priors, and datasets have been investigated. In this paper, we provide a comprehensive survey of learning-based camera calibration techniques, by analyzing their strengths and limitations. Our main calibration categories include the standard pinhole camera model, distortion camera model, cross-view model, and cross-sensor model, following the research trend and extended applications. As there is no unified benchmark in this community, we collect a holistic calibration dataset that can serve as a public platform to evaluate the generalization of existing methods. It comprises both synthetic and real-world data, with images and videos captured by different cameras in diverse scenes. Toward the end of this paper, we discuss the challenges and provide further research directions. To our knowledge, this is the first survey for the learning-based camera calibration (spanned 10 years). The summarized methods, datasets, and benchmarks are available and will be regularly updated at https://github.com/KangLiao929/Awesome-Deep-Camera-Calibration.

Problem

• Surveys learning-based camera calibration techniques
• Analyzes strengths and limitations of methods
• Proposes a unified benchmark for evaluation

Innovation

• Learning-based camera calibration techniques
• Comprehensive survey of calibration models
• Holistic dataset for method evaluation