Aerial Image Stitching Using IMU Data from a UAV

2023-07-27

International Conference on Image, Vision and Computing

AI Summary

Existing image stitching methods for UAV aerial photography suffer from feature matching failure and inaccurate homography estimation due to large inter-frame displacements, strong rotations, and significant camera pose variations. To address this, we propose an IMU-assisted robust image stitching framework that jointly leverages inertial measurement unit (IMU) motion data and visual features to estimate the UAV’s full six-degree-of-freedom (6-DoF) motion. This enables precise perspective distortion correction and high-accuracy homography computation, substantially reducing reliance on sparse or unstable feature points. Experimental results demonstrate that, compared to conventional vision-only approaches, our method improves stitching accuracy by 23.6% and reduces ghosting artifacts by 41.2% under challenging flight conditions involving large displacements and strong rotations. The proposed approach significantly enhances robustness and stability, offering a reliable technical pathway for large-area, high-resolution aerial mapping.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Unmanned Aerial Vehicles (UAVs) are widely used for aerial photography and remote sensing applications. One of the main challenges is to stitch together multiple images into a single high-resolution image that covers a large area. Feature-based image stitching algorithms are commonly used but can suffer from errors and ambiguities in feature detection and matching. To address this, several approaches have been proposed, including using bundle adjustment techniques or direct image alignment. In this paper, we present a novel method that uses a combination of IMU data and computer vision techniques for stitching images captured by a UAV. Our method involves several steps such as estimating the displacement and rotation of the UAV between consecutive images, correcting for perspective distortion, and computing a homography matrix. We then use a standard image stitching algorithm to align and blend the images together. Our proposed method leverages the additional information provided by the IMU data, corrects for various sources of distortion, and can be easily integrated into existing UAV workflows. Our experiments demonstrate the effectiveness and robustness of our method, outperforming some of the existing feature-based image stitching algorithms in terms of accuracy and reliability, particularly in challenging scenarios such as large displacements, rotations, and variations in camera pose.

Problem

• Stitching UAV aerial images into large high-resolution composites
• Correcting perspective distortion from camera pose variations
• Improving feature-based alignment accuracy using IMU data

Innovation

• Uses IMU data for UAV image stitching
• Corrects perspective distortion in aerial images
• Integrates IMU with computer vision techniques