FrogDogNet: Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensing

2025-04-23

AI Summary

To address misclassification in cross-domain zero-shot classification of remote sensing images using CLIP—caused by intra-class variation and background noise—this paper proposes a visual prompt learning framework integrating Fourier-domain low-frequency preservation with self-attention mechanisms. The method explicitly retains domain-invariant low-frequency structural features via frequency-domain filtering, while self-attention dynamically focuses on discriminative image regions, overcoming the vulnerability of conventional full-image prompting to contextual interference. Evaluated on four remote sensing benchmarks across three challenging domain generalization settings—cross-sensor, cross-season, and cross-region—the approach consistently outperforms state-of-the-art methods, achieving significant gains in zero-shot classification accuracy. Notably, it establishes the first principled integration of spectral-domain modeling with vision-language prompt learning, introducing a novel paradigm for robust domain generalization in remote sensing vision-language models.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In recent years, large-scale vision-language models (VLMs) like CLIP have gained attention for their zero-shot inference using instructional text prompts. While these models excel in general computer vision, their potential for domain generalization in remote sensing (RS) remains underexplored. Existing approaches enhance prompt learning by generating visual prompt tokens but rely on full-image features, introducing noise and background artifacts that vary within a class, causing misclassification. To address this, we propose FrogDogNet, a novel prompt learning framework integrating Fourier frequency filtering and self-attention to improve RS scene classification and domain generalization. FrogDogNet selectively retains invariant low-frequency components while eliminating noise and irrelevant backgrounds, ensuring robust feature representation across domains. The model first extracts significant features via projection and self-attention, then applies frequency-based filtering to preserve essential structural information for prompt learning. Extensive experiments on four RS datasets and three domain generalization tasks show that FrogDogNet consistently outperforms state-of-the-art prompt learning methods, demonstrating superior adaptability across domain shifts. Our findings highlight the effectiveness of frequency-based invariant feature retention in generalization, paving the way for broader applications. Our code is available at https://github.com/HariseetharamG/FrogDogNet

Problem

• Enhance domain generalization in remote sensing using CLIP
• Reduce noise and background artifacts in visual prompts
• Improve RS scene classification via frequency-based filtering

Innovation

• Fourier frequency filtering retains invariant features
• Self-attention enhances significant feature extraction
• Frequency-based filtering removes noise and backgrounds