Seeing Across Skies and Streets: Feedforward 3D Reconstruction from Satellite, Drone, and Ground Images

2026-05-08

AI Summary

This work addresses the limitation of existing cross-view localization methods, which typically estimate only 3-degree-of-freedom (3-DoF) poses and struggle with terrain undulations and camera tilts in real-world scenarios. To overcome this, we propose Cross3R—an end-to-end feedforward neural network that, for the first time, leverages drone imagery without known relative poses as an intermediate view to jointly fuse satellite and ground-level images. Within a single forward pass, Cross3R simultaneously achieves 6-DoF camera pose estimation, geolocalization, and 3D point cloud reconstruction. To enable this research, we introduce CrossGeo, a large-scale triview dataset comprising 278,000 images. Experiments demonstrate that Cross3R significantly outperforms existing feedforward 3D methods on CrossGeo and surpasses cross-view models specifically designed for KITTI, highlighting its strong generalization capability.

Abstract

Cross-view localization classically asks: where does this ground image lie on the satellite tile? Existing methods are typically limited to 3-DoF estimates -- an $(x,y)$ position and a yaw angle -- because nadir satellite imagery provides no direct cues for roll, pitch, or altitude, forcing a reliance on planar-motion and zero-tilt assumptions. These assumptions break on real terrain with slopes, ramps, and tilted camera mounts. To overcome this, we introduce a single UAV image as an intermediate viewpoint: it reveals the 3D structure invisible from nadir, supplies the cues for roll, pitch, and altitude that the satellite alone cannot provide, and needs only spatial overlap with the ground camera -- no known relative pose is required. Building on this insight, we propose **Cross3R**, a flexible feed-forward model that ingests a satellite tile together with a UAV image, a ground image, or both, and, in a single forward pass, recovers a cross-view 3D point cloud, the 6-DoF poses of every input camera, and the on-tile $(x,y)$ position and yaw of each perspective camera. For training and evaluation, we also construct **CrossGeo**, a 278K-image tri-view dataset spanning 85 scenes across every continent except Antarctica. On CrossGeo, Cross3R consistently outperforms feed-forward 3D baselines in point-cloud reconstruction, 6-DoF camera-pose estimation, and cross-view localization. On KITTI, it outperforms dedicated cross-view methods trained on KITTI on most metrics, despite having no KITTI training itself.

Problem

• cross-view localization
• 3D reconstruction
• 6-DoF pose estimation
• satellite imagery
• UAV imagery

Innovation

• Cross-view localization
• 3D reconstruction
• 6-DoF pose estimation
• UAV imagery
• feedforward model