Robust structure from motion for aerial-ground images via detector-free feature matching and multi-view track refinement

2026-08-15

AI Summary

This study addresses feature matching challenges in aerial image 3D reconstruction caused by viewpoint, scale, and rotation variations. We propose a rotation-robust detector-free matching network coupled with multi-view trajectory optimization. Specifically, an omnidirectional state space block enables rotation-invariant feature extraction, while quadtree attention and bidirectional coarse-to-fine mechanisms refine correspondence. Additionally, a multi-view trajectory optimization strategy is designed to enhance feature repeatability. Experimental results demonstrate that under 5° pose error, the proposed method achieves a 93.9% AUC improvement over LoFTR and increases incremental Structure-from-Motion reconstruction accuracy by 27.6%–32.7%, significantly enhancing both system stability and precision.

Abstract

Integrated 3D reconstruction from aerial-ground images is essential for generating high-precision urban 3D models, yet severe variations in viewpoint, scale, and rotation make robust feature matching highly challenging. To address these limitations, this study introduces a rotation-robust detector-free matching network coupled with multi-view track refinement for incremental Structure from Motion (ISfM). The proposed workflow features four key modules. First, rotation-aware feature extraction replaces traditional convolutions with an Omnidirectional State Space Block (OSS Block) that selectively scans across eight symmetrical directions to model long-range spatial dependencies and synthesize rotation-invariant feature maps. Second, multi-scale attention transformation utilizes quadtree attention to build a hierarchical token pyramid that isolates high-association token regions and discards irrelevant areas, capturing long-range context with linear computational complexity. Third, bi-directional feature matching executes a symmetric coarse-to-fine matching scheme where coarse alignment computes dual-direction Softmax confidence matrices under mutual nearest neighbor constraints, and fine alignment uses a multi-layer perceptron to regress sub-pixel coordinate offsets. Finally, multi-view track refinement employs an integrated indexing structure to evaluate localized spatial proximity and link disjoint sub-tracks to the highest-confidence anchor point, ensuring stable feature repeatability across the ISfM pipeline. By using real aerial-ground datasets, experimental results demonstrate that the proposed method improves AUC at 5° pose error by 93.9% compared with LoFTR and achieves the highest precision in ISfM reconstruction, with the improved accuracy ranging from 27.6% to 32.7%. The proposed method provides a reliable solution for integrated 3D reconstruction of aerial-ground images.

Problem

• Aerial-ground images
• Structure from Motion
• Feature matching
• 3D reconstruction

Innovation

• Detector-free Matching
• Omnidirectional State Space Block
• Quadtree Attention
• Multi-view Track Refinement
• Aerial-ground SfM