FlameVQA: A Physically-Grounded UAV Wildfire VQA Benchmark with Radiometric Thermal Supervision

2026-06-25

AI Summary

This work addresses the limitations of RGB-based visual perception in unmanned aerial vehicle (UAV) wildfire monitoring—particularly its susceptibility to smoke, scale variation, and occlusion—and the absence of benchmarks that integrate thermal imaging for safety-critical reasoning. To bridge this gap, the authors construct the first multimodal visual question answering benchmark leveraging the FLAME 3 dataset, which combines RGB images with radiometric thermal imaging TIFFs. The benchmark incorporates thermodynamic physical constraints and multimodal consistency checks, with annotations refined through multimodal large language models, rule-based validation, and human review to ensure reliability. It defines six capability dimensions tailored to emergency response scenarios. Evaluations on the open-sourced dataset and code reveal that current multimodal large language models perform adequately when explicit cross-modal cues are present but fail significantly in dense smoke conditions on tasks such as target existence verification and fire coverage estimation, underscoring the need for domain-specific adaptation.

Abstract

Wildfire monitoring from UAVs requires reliable reasoning over complex aerial scenes, where smoke, scale variation, and occlusions often limit RGB-only interpretation. We introduce FlameVQA, a multiple-choice visual question answering benchmark for UAV-based wildfire intelligence built on FLAME 3, leveraging paired RGB imagery and radiometric thermal TIFFs for temperature-grounded, safety-critical reasoning. FlameVQA includes 34 multiple-choice questions per image spanning six operational capability groups, covering tasks such as detection, localization, distribution/coverage estimation, cross-modal reasoning, and flight planning. To ensure label reliability, we combine MLLM-assisted annotation with deterministic thermal rules and cross-question consistency checks, followed by human auditing. We also evaluate representative MLLMs on FlameVQA to provide baselines for future work. Results show strong performance when explicit cross-modal cues are available, but notable failures on presence detection under heavy smoke and on coverage estimation. These findings suggest that current MLLMs require domain-specific adaptation to better support disaster and wildfire monitoring. The dataset and benchmark code are open-source at github.com/mobiiin/WildFire_VQA

Problem

• wildfire monitoring
• UAV
• visual question answering
• thermal imaging
• cross-modal reasoning

Innovation

• radiometric thermal supervision
• UAV wildfire monitoring
• visual question answering
• cross-modal reasoning
• physically-grounded VQA