WildBox: A Dataset and Benchmark for Aerial Monocular 3D Detection of African Savanna Wildlife

2026-06-19

AI Summary

This study addresses the challenge of monocular 3D detection of savanna wildlife in drone-captured aerial imagery by introducing WildBox, a large-scale dataset comprising 237,505 3D bounding boxes across seven animal species and six benchmark categories. The work identifies depth estimation as the primary bottleneck in open-vocabulary monocular 3D detection—a finding previously unreported. Leveraging coarse-to-fine curriculum learning and architectures such as OVMono3D-LIFT and DetAny3D, evaluations are conducted in a KITTI/Omni3D-compatible scale-normalized coordinate system. Experimental results reveal zero-shot performance of 0.00 AP, while fine-tuning achieves 8.68 ± 0.47 AP-BEV@0.50 and 13.17 ± 0.69 AP3D macro. Notably, depth estimation accounts for over 84% of the normalized Hausdorff distance, underscoring its critical role in detection accuracy.

Abstract

We introduce WildBox, a dataset and benchmark for monocular 3D detection of wildlife from drone video, comprising 237,505 3D bounding box annotations across seven African savanna species grouped into six benchmark classes. Annotations follow a KITTI/Omni3D-compatible format in a per-segment scale-normalised camera frame, with instance identities maintained across each segment. We evaluate two open-vocabulary monocular 3D architectures, OVMono3D-LIFT and DetAny3D, under zero-shot, ground-truth 2D box prompt, and supervised fine-tuning protocols. Open-vocabulary 2D foundation models provide usable zero-shot wildlife localisation (50.55 AP@50), but zero-shot 3D detection collapses to 0.00 AP across both architectures and every 2D-input condition tested, including ground-truth 2D box prompts, thus isolating the failure to the 3D stage. Fine-tuning on WildBox recovers performance to 8.68 +/- 0.47 AP-BEV@0.50 and 13.17 +/- 0.69 AP3D macro. Depth contributes 84% of normalised Hausdorff distance after fine-tuning and over 99% in zero-shot, identifying monocular aerial depth as the dominant open problem in this regime. A coarse-to-fine curriculum, i.e. pretraining on a merged zebra class before fine-tuning on the Grevy's/plains split, improves macro 3D performance with less total compute, with the largest gains on the two zebra subclasses. WildBox is released with video-level splits, evaluation code, and baseline checkpoints to enable progress in 3D wildlife perception from drone video.

Problem

• monocular 3D detection
• aerial imagery
• wildlife monitoring
• depth estimation
• zero-shot learning

Innovation

• monocular 3D detection
• aerial wildlife dataset
• zero-shot 3D perception
• depth estimation bottleneck
• coarse-to-fine curriculum