CNN-Based Camera Pose Estimation and Localisation of Scan Images for Aircraft Visual Inspection

2025-11-23

AI Summary

To address the challenge of camera pose estimation and image localization for external visual inspection of commercial aircraft under infrastructure-free, non-contact, and non-UAV conditions, this paper proposes a field-deployable end-to-end automated method. The approach directly regresses the full six-degree-of-freedom camera pose using a deep CNN fine-tuned exclusively on synthetic imagery—marking the first such application. Domain randomization is incorporated to improve cross-domain generalization, and a constraint loss function is designed to embed prior knowledge of aircraft geometric structure. The method requires no calibration targets, GPS, or auxiliary sensors, enabling rapid initialization and precise panoramic zoom-image localization in uncontrolled environments such as airport gate areas. Experimental evaluation on real-world scenes achieves root-mean-square pose estimation errors of 0.24 m and 2°, demonstrating substantial improvements in automation capability and deployment flexibility for aircraft inspection workflows.

Technology Category

• Computer Vision: Biometrics, Face, Gesture & Pose
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Security and Privacy: Browser and end-point security

Abstract

General Visual Inspection is a manual inspection process regularly used to detect and localise obvious damage on the exterior of commercial aircraft. There has been increasing demand to perform this process at the boarding gate to minimise the downtime of the aircraft and automating this process is desired to reduce the reliance on human labour. Automating this typically requires estimating a camera's pose with respect to the aircraft for initialisation but most existing localisation methods require infrastructure, which is very challenging in uncontrolled outdoor environments and within the limited turnover time (approximately 2 hours) on an airport tarmac. Additionally, many airlines and airports do not allow contact with the aircraft's surface or using UAVs for inspection between flights, and restrict access to commercial aircraft. Hence, this paper proposes an on-site method that is infrastructure-free and easy to deploy for estimating a pan-tilt-zoom camera's pose and localising scan images. This method initialises using the same pan-tilt-zoom camera used for the inspection task by utilising a Deep Convolutional Neural Network fine-tuned on only synthetic images to predict its own pose. We apply domain randomisation to generate the dataset for fine-tuning the network and modify its loss function by leveraging aircraft geometry to improve accuracy. We also propose a workflow for initialisation, scan path planning, and precise localisation of images captured from a pan-tilt-zoom camera. We evaluate and demonstrate our approach through experiments with real aircraft, achieving root-mean-square camera pose estimation errors of less than 0.24 m and 2 degrees for all real scenes.

Problem

• Automating aircraft visual inspection without infrastructure requirements
• Estimating camera pose in uncontrolled outdoor airport environments
• Localizing scan images using PTZ cameras with deep learning

Innovation

• Uses CNN for camera pose estimation
• Employs synthetic images with domain randomization
• Leverages aircraft geometry to improve accuracy