Towards Autonomous UAV Visual Object Search in City Space: Benchmark and Agentic Methodology
🤖 AI Summary
To address challenges in autonomous visual object search (AVOS) for drones in urban environments—including semantic redundancy, confusion among visually similar objects, and imbalance between exploration and exploitation—this paper introduces CityAVOS, the first city-scale AVOS benchmark. We propose PRPSearcher, an intelligent agent framework implementing a three-tier cognitive search pipeline: perception, reasoning, and planning. Our method features a novel tri-map coordination mechanism integrating a dynamic semantic map, a 3D cognitive map, and an uncertainty map; further enhanced by Inspiration Promote Thought (IPT) prompting and a similarity-aware denoising module to improve semantic understanding and navigation robustness. Experiments on CityAVOS demonstrate significant improvements over baselines: a 37.69% increase in success rate, a 28.96% gain in Success-weighted by Path Length (SPL), a 30.69% reduction in average search steps, and a 46.40% decrease in navigation error.
Application Category
📝 Abstract
Aerial Visual Object Search (AVOS) tasks in urban environments require Unmanned Aerial Vehicles (UAVs) to autonomously search for and identify target objects using visual and textual cues without external guidance. Existing approaches struggle in complex urban environments due to redundant semantic processing, similar object distinction, and the exploration-exploitation dilemma. To bridge this gap and support the AVOS task, we introduce CityAVOS, the first benchmark dataset for autonomous search of common urban objects. This dataset comprises 2,420 tasks across six object categories with varying difficulty levels, enabling comprehensive evaluation of UAV agents' search capabilities. To solve the AVOS tasks, we also propose PRPSearcher (Perception-Reasoning-Planning Searcher), a novel agentic method powered by multi-modal large language models (MLLMs) that mimics human three-tier cognition. Specifically, PRPSearcher constructs three specialized maps: an object-centric dynamic semantic map enhancing spatial perception, a 3D cognitive map based on semantic attraction values for target reasoning, and a 3D uncertainty map for balanced exploration-exploitation search. Also, our approach incorporates a denoising mechanism to mitigate interference from similar objects and utilizes an Inspiration Promote Thought (IPT) prompting mechanism for adaptive action planning. Experimental results on CityAVOS demonstrate that PRPSearcher surpasses existing baselines in both success rate and search efficiency (on average: +37.69% SR, +28.96% SPL, -30.69% MSS, and -46.40% NE). While promising, the performance gap compared to humans highlights the need for better semantic reasoning and spatial exploration capabilities in AVOS tasks. This work establishes a foundation for future advances in embodied target search. Dataset and source code are available at https://anonymous.4open.science/r/CityAVOS-3DF8.
Problem
Research questions and friction points this paper is trying to address.
Autonomous UAV visual object search in urban environments
Challenges in semantic processing and object distinction
Need for improved exploration-exploitation balance in AVOS
Innovation
Methods, ideas, or system contributions that make the work stand out.
Multi-modal LLM-powered agentic method PRPSearcher
Three specialized maps for enhanced UAV cognition
Denoising and IPT mechanisms for adaptive planning
🔎 Similar Papers
No similar papers found.
