Seeing through Satellite Images at Street Views

2025-05-22

AI Summary

This work addresses the challenging cross-view synthesis problem from satellite imagery to street-level 360° panoramas (including video). We propose Sat2Density++, the first NeRF-based method that explicitly incorporates street-scene-specific priors—such as sky distribution and global illumination—into the neural radiance field framework, enabling explicit viewpoint disentanglement and joint geometry-appearance optimization. By integrating these priors, our approach effectively mitigates radiance field misalignment caused by extreme viewpoint discrepancies and sparse input observations. Evaluated on urban and suburban datasets, Sat2Density++ generates high-fidelity, multi-view-consistent 360° street-level images and videos. Compared to existing cross-view synthesis methods, it achieves significant improvements in rendering quality and satellite-image fidelity. This work establishes a novel paradigm for synergistic modeling of remote sensing and street-level imagery.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

This paper studies the task of SatStreet-view synthesis, which aims to render photorealistic street-view panorama images and videos given any satellite image and specified camera positions or trajectories. We formulate to learn neural radiance field from paired images captured from satellite and street viewpoints, which comes to be a challenging learning problem due to the sparse-view natural and the extremely-large viewpoint changes between satellite and street-view images. We tackle the challenges based on a task-specific observation that street-view specific elements, including the sky and illumination effects are only visible in street-view panoramas, and present a novel approach Sat2Density++ to accomplish the goal of photo-realistic street-view panoramas rendering by modeling these street-view specific in neural networks. In the experiments, our method is testified on both urban and suburban scene datasets, demonstrating that Sat2Density++ is capable of rendering photorealistic street-view panoramas that are consistent across multiple views and faithful to the satellite image.

Problem

• Render street-view panoramas from satellite images
• Overcome large viewpoint changes between satellite and street views
• Model sky and illumination effects for realistic rendering

Innovation

• Neural radiance field for satellite-street view synthesis
• Modeling sky and illumination effects in neural networks
• Sat2Density++ for photorealistic street-view panorama rendering