Scale-Aware Recognition in Satellite Images under Resource Constraint

2024-10-31

arXiv.org

AI Summary

To address the conflict between object-scale diversity in satellite imagery and constrained computational resources, this paper proposes a scale-adaptive recognition framework. Methodologically, it introduces large language models (LLMs) for the first time to perform semantic-driven scale concept reasoning; designs an active high-resolution (HR) image sampling strategy based on model disagreement; and establishes an HR–low-resolution (LR) cross-resolution knowledge distillation mechanism. The core innovation lies in jointly modeling LLM priors, model uncertainty, and multi-scale representations to enable budget-aware, on-demand HR invocation. Under stringent resource constraints, the framework achieves a 26.3% improvement in recognition accuracy over the full-HR baseline while reducing HR image usage by 76.3%, significantly enhancing deployment efficiency and cost-effectiveness.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recognition of features in satellite imagery (forests, swimming pools, etc.) depends strongly on the spatial scale of the concept and therefore the resolution of the images. This poses two challenges: Which resolution is best suited for recognizing a given concept, and where and when should the costlier higher-resolution (HR) imagery be acquired? We present a novel scheme to address these challenges by introducing three components: (1) A technique to distill knowledge from models trained on HR imagery to recognition models that operate on imagery of lower resolution (LR), (2) a sampling strategy for HR imagery based on model disagreement, and (3) an LLM-based approach for inferring concept"scale". With these components we present a system to efficiently perform scale-aware recognition in satellite imagery, improving accuracy over single-scale inference while following budget constraints. Our novel approach offers up to a 26.3% improvement over entirely HR baselines, using 76.3% fewer HR images.

Problem

• Satellite Imagery
• Object Recognition
• Resource Optimization

Innovation

• Knowledge Transfer
• High-Resolution Sampling
• Scale-aware Recognition