Iterative Motion Compensation for Canonical 3D Reconstruction from UAV Plant Images Captured in Windy Conditions

2025-10-17

AI Summary

Motion blur and misalignment in UAV-captured plant imagery—induced by wind-induced foliage motion—severely degrade 3D reconstruction accuracy. Method: We propose an iterative motion-compensated 3D reconstruction framework integrating ArUco marker–aided pose estimation, optical-flow–driven image alignment, differentiable rendering–guided leaf deformation optimization, and autonomous UAV flight control. Leveraging intermediate reconstructions, the method iteratively refines dynamic leaf geometry to achieve motion-robust multi-view image standardization. Contribution/Results: Our approach significantly improves reconstruction fidelity and mesh resolution over state-of-the-art methods. It is rigorously validated across diverse crops and time points in real-world field conditions. To foster reproducibility and community advancement, we publicly release all source code and introduce the first multimodal UAV image dataset specifically designed for dynamic plant reconstruction.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

3D phenotyping of plants plays a crucial role for understanding plant growth, yield prediction, and disease control. We present a pipeline capable of generating high-quality 3D reconstructions of individual agricultural plants. To acquire data, a small commercially available UAV captures images of a selected plant. Apart from placing ArUco markers, the entire image acquisition process is fully autonomous, controlled by a self-developed Android application running on the drone's controller. The reconstruction task is particularly challenging due to environmental wind and downwash of the UAV. Our proposed pipeline supports the integration of arbitrary state-of-the-art 3D reconstruction methods. To mitigate errors caused by leaf motion during image capture, we use an iterative method that gradually adjusts the input images through deformation. Motion is estimated using optical flow between the original input images and intermediate 3D reconstructions rendered from the corresponding viewpoints. This alignment gradually reduces scene motion, resulting in a canonical representation. After a few iterations, our pipeline improves the reconstruction of state-of-the-art methods and enables the extraction of high-resolution 3D meshes. We will publicly release the source code of our reconstruction pipeline. Additionally, we provide a dataset consisting of multiple plants from various crops, captured across different points in time.

Problem

• Mitigating wind-induced plant motion in UAV imaging
• Improving 3D reconstruction accuracy under windy conditions
• Compensating leaf movement through iterative image deformation

Innovation

• Iterative motion compensation using optical flow
• Gradual image deformation for wind correction
• Autonomous UAV data acquisition with Android control