WildLIFT: Lifting monocular drone video to 3D for species-agnostic wildlife monitoring

2026-04-27

AI Summary

This work addresses the limitation of existing monocular drone-based wildlife monitoring approaches, which predominantly rely on 2D imagery and neglect 3D geometric information. The authors propose a species-agnostic 3D detection and tracking framework that integrates monocular video-based 3D reconstruction with open-vocabulary 2D instance segmentation to produce oriented 3D bounding boxes enriched with semantic orientation. This is the first method to enable species-independent 3D perception for wildlife monitoring, facilitating viewpoint coverage and occlusion quantification analyses. By incorporating keyframe optimization, the approach significantly reduces annotation costs. Evaluated on a dataset comprising 2,581 frames with over 6,700 3D instances across four large mammal species, the framework achieves high identity consistency in multi-object 3D tracking, substantially enhancing the structural richness of ecological data for downstream analysis.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Monocular RGB cameras mounted on drones are widely used for wildlife monitoring, yet most analytical pipelines remain confined to two-dimensional image space, leaving geometric information in video underexploited. We present WildLIFT, a computational framework that integrates three-dimensional scene geometry from monocular drone video with open-vocabulary 2D instance segmentation to enable species-agnostic 3D detection and tracking. Oriented 3D bounding box labels with semantic face information enable quantitative assessment of viewpoint coverage and inter-animal occlusion, producing structured metadata for downstream ecological analyses. We validate the framework on 2,581 manually curated frames comprising over 6,700 3D detections across four large mammal species. WildLIFT maintains high identity consistency in multi-animal scenes and substantially reduces manual 3D annotation effort through keyframe-based refinement. By transforming standard drone footage into structured 3D and viewpoint-aware representations, WildLIFT extends the analytical utility of aerial wildlife datasets for behavioural research and population monitoring.

Problem

• wildlife monitoring
• monocular drone video
• 3D detection
• species-agnostic
• scene geometry

Innovation

• monocular 3D reconstruction
• species-agnostic detection
• drone-based wildlife monitoring
• oriented 3D bounding boxes
• open-vocabulary segmentation