Toward Active Object Detection for UAVs in the Wild: A Large-Scale Dataset, Benchmark and Method

2026-07-09

AI Summary

This work addresses the challenges of occlusion and sparse target pixels that commonly hinder object detection by drones in real-world environments. Existing active detection approaches are limited by the lack of large-scale, high-quality datasets and insufficient generalization capabilities. To bridge this gap, the authors introduce ATRNet-LUDO, the first large-scale real-world drone-to-ground active object detection dataset, along with a corresponding benchmark. They further propose AOD-JEPA, a novel method that integrates task-specific priors into a Joint Embedding Predictive Architecture (JEPA) to learn robust state representations and enhance policy generalization. Experimental results demonstrate that AOD-JEPA significantly outperforms current methods on the proposed benchmark, validating the efficacy of the introduced world model.

Abstract

Object detection is a fundamental component in numerous Unmanned Aerial Vehicle (UAV) applications, yet it has long been plagued by hindrances like occlusion or target pixel scarcity. Active Object Detection (AOD) provides a novel paradigm to address these challenges via active vision, while UAV-based AOD research remains scarce due to the lack of high-quality datasets and benchmarks for algorithm development and evaluation. To fill this gap, this paper presents ATRNet-LUDO, the first large-scale real-world dataset for UAV-Ground Active Object Detection (UGAOD). It contains 121,000 multi-view panoramic multi-target aerial images and 1.21 million local single-target slices, covering 10 vehicle targets across 40 scenarios. It enables the construction of diverse training and testing environments for UAV agent interaction and active observation policy learning. Based on this dataset, we establish a comprehensive evaluation benchmark for AOD policy learning methods. Most existing AOD policies rely on Deep Reinforcement Learning (DRL) but suffer from poor generalization. Evaluations on our benchmark reveal a significant generalization gap between training and testing performance, highlighting an urgent need for solutions. To this end, we leverage the Joint Embedding Predictive Architecture (JEPA) to construct a world model that enhances state representation learning, and propose AOD-JEPA by incorporating AOD-specific prior knowledge. Extensive experiments validate its effectiveness and superiority. We hope ATRNet-LUDO and the benchmark will advance research in the UGAOD field. The dataset and code are soon available at https://github.com/Leo000ooo/LUDO_dataset.

Problem

• Active Object Detection
• UAV
• Dataset
• Benchmark
• Generalization

Innovation

• Active Object Detection
• UAV
• Large-scale Dataset
• JEPA
• Generalization