Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System

2026-08-12

AI Summary

This study addresses the lack of a unified benchmark for unmanned aerial vehicle (UAV) visual question answering (VQA) and the limited performance of multimodal large language models (MLLMs) under challenging conditions such as extreme scales, arbitrary viewpoints, and high-density targets. To this end, the authors introduce UAVQA-Bench, the first comprehensive benchmark spanning six dimensions and sixteen tasks, comprising 1,500 question-answer pairs. They further propose UAV-MAS, a training-free multi-agent system that mitigates three key failure modes—domain-tool mismatch, error propagation, and static reasoning—through domain-aware tool scheduling, iterative contextual validation, and difficulty-adaptive search. Evaluated on UAVQA-Bench, UAV-MAS achieves 77.0% accuracy using a 32B open-source MLLM, outperforming Gemini 3 Pro by 4.0%; its 8B variant improves over the baseline by 8.7%.

Abstract

Multimodal Large Language Model (MLLM)-based UAV aerial image understanding and reasoning is essential for aerial intelligence yet poses distinct challenges arising from extreme scale variation, arbitrary camera orientations, and high object density. Despite growing interest, existing evaluations remain fragmented across individual datasets and narrow tasks, leaving a critical gap in unified assessment of UAV understanding and reasoning capabilities. To fill this gap, we construct UAVQA-Bench, a benchmark of 1,500 human-annotated QA pairs drawn from 13 public UAV datasets, covering 6 capability dimensions and 16 tasks in both multiple-choice and visual grounding formats. Systematic evaluation of a broad range of open-source and closed-source MLLMs as well as agent-based systems on UAVQA-Bench identifies three key failure modes: domain-toolset mismatch, unchecked error propagation, and static reasoning. Motivated by these findings, we propose UAV-MAS, a training-free multi-agent system for MLLM-based UAV aerial image understanding and reasoning, comprising a Domain-Specific Perception Engine (DSPE) that routes queries to task-appropriate visual tools, a Context-Aware Iterative Refinement module (CAIR) that validates intermediate reasoning to curb error accumulation, and a Difficulty-Aware Adaptive Search mechanism (DAAS) that adjusts search depth to question difficulty. UAV-MAS with a 32B open-source MLLM achieves 77.0% overall accuracy on UAVQA-Bench, surpassing Gemini 3 Pro by 4.0\%, while the 8B variant improves 8.7\% over its base model.

Problem

• UAV image understanding
• Multimodal Large Language Model
• benchmark
• reasoning
• aerial intelligence

Innovation

• UAVQA-Bench
• UAV-MAS
• training-free multi-agent system
• Domain-Specific Perception Engine
• Difficulty-Aware Adaptive Search