Knowledge-aware Visual Question Generation for Remote Sensing Images

2026-02-22

AI Summary

Existing methods for remote sensing visual question generation often suffer from template-driven formulations and shallow semantics, limiting their utility in practical question-answering or dialogue systems. To address this, this work proposes KRSVQG, a knowledge-aware model that, for the first time, incorporates external domain-specific knowledge triples into the task. By leveraging image captions as an intermediate representation, KRSVQG effectively fuses remote sensing image features with structured knowledge to enable knowledge-guided question generation. Experimental results on the human-annotated NWPU-300 and TextRS-300 datasets demonstrate that KRSVQG significantly outperforms current state-of-the-art approaches, generating questions with markedly improved semantic richness, knowledge relevance, and image alignment.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Natural Language Processing: Question Answering

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web question answering
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

With the rapid development of remote sensing image archives, asking questions about images has become an effective way of gathering specific information or performing image retrieval. However, automatically generated image-based questions tend to be simplistic and template-based, which hinders the real deployment of question answering or visual dialogue systems. To enrich and diversify the questions, we propose a knowledge-aware remote sensing visual question generation model, KRSVQG, that incorporates external knowledge related to the image content to improve the quality and contextual understanding of the generated questions. The model takes an image and a related knowledge triplet from external knowledge sources as inputs and leverages image captioning as an intermediary representation to enhance the image grounding of the generated questions. To assess the performance of KRSVQG, we utilized two datasets that we manually annotated: NWPU-300 and TextRS-300. Results on these two datasets demonstrate that KRSVQG outperforms existing methods and leads to knowledge-enriched questions, grounded in both image and domain knowledge.

Problem

• Visual Question Generation
• Remote Sensing Images
• Knowledge-aware
• Image Grounding
• Question Diversity

Innovation

• knowledge-aware
• visual question generation
• remote sensing images
• image captioning
• knowledge triplet