AnimalClue: Recognizing Animals by their Traces

2025-07-27

AI Summary

This study addresses the challenge of species identification from wildlife indirect evidence—such as footprints, scat, eggs, bones, and feathers—by introducing the first large-scale, fine-grained, multimodal image dataset dedicated to such traces. Comprising 159,605 pixel-accurate bounding boxes across 968 species, the dataset provides both species-level labels and morphological trait annotations. Methodologically, we systematically benchmark state-of-the-art object detection, instance segmentation, and classification models on their ability to recognize subtle, low-salience visual cues amid complex textures. Experimental results reveal significant performance limitations of current vision models, highlighting fundamental challenges in cross-species trace recognition. Our contribution is twofold: (1) filling a critical data gap for intelligent indirect-evidence analysis, and (2) establishing a rigorous, community-standard benchmark to advance biodiversity monitoring and interspecific interaction studies grounded in fine-grained visual feature analysis.

Technology Category

• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data
• Machine Learning: Multimodal Learning

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Wildlife observation plays an important role in biodiversity conservation, necessitating robust methodologies for monitoring wildlife populations and interspecies interactions. Recent advances in computer vision have significantly contributed to automating fundamental wildlife observation tasks, such as animal detection and species identification. However, accurately identifying species from indirect evidence like footprints and feces remains relatively underexplored, despite its importance in contributing to wildlife monitoring. To bridge this gap, we introduce AnimalClue, the first large-scale dataset for species identification from images of indirect evidence. Our dataset consists of 159,605 bounding boxes encompassing five categories of indirect clues: footprints, feces, eggs, bones, and feathers. It covers 968 species, 200 families, and 65 orders. Each image is annotated with species-level labels, bounding boxes or segmentation masks, and fine-grained trait information, including activity patterns and habitat preferences. Unlike existing datasets primarily focused on direct visual features (e.g., animal appearances), AnimalClue presents unique challenges for classification, detection, and instance segmentation tasks due to the need for recognizing more detailed and subtle visual features. In our experiments, we extensively evaluate representative vision models and identify key challenges in animal identification from their traces. Our dataset and code are available at https://dahlian00.github.io/AnimalCluePage/

Problem

• Identifying species from indirect evidence like footprints and feces
• Lack of large-scale datasets for animal trace recognition
• Challenges in classifying subtle visual features in wildlife traces

Innovation

• Large-scale dataset for indirect evidence identification
• Covers 968 species with detailed annotations
• Evaluates vision models on subtle visual features