Deep learning-based ecological analysis of camera trap images is impacted by training data quality and quantity

2024-08-26

AI Summary

This study investigates how label noise and dataset size reduction in deep learning–based automated annotation of camera-trap images affect ecological inference. Using datasets from African savannas and Asian subtropical dry forests, we trained ResNet and EfficientNet models under controlled Monte Carlo–style label noise (≤10%) and subsampling (≤50%), then evaluated robustness in estimating species richness, occupancy, and activity rhythms. We find that community-level metrics remain highly robust—errors remain below 5%—whereas occupancy estimates for rare species degrade by a factor of 3.2, and diel activity peak timing shifts by up to 2.8 hours. Critically, model architecture does not influence the magnitude or direction of ecological bias, indicating that annotation quality—not model choice—drives inference accuracy. This is the first quantitative demonstration that high-fidelity, balanced labeling is essential for reliable quantitative conservation ecology.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Bias, Fairness & Privacy
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

Large image collections generated from camera traps offer valuable insights into species richness, occupancy, and activity patterns, significantly aiding biodiversity monitoring. However, the manual processing of these datasets is time-consuming, hindering analytical processes. To address this, deep neural networks have been adopted to automate image labelling, but the impact of classification error on ecological metrics remains unclear. Here, we analyse data from camera trap collections in an African savannah (82,300 images, 47 species) and an Asian sub-tropical dry forest (40,308 images, 29 species) to compare ecological metrics derived from expert-generated species identifications with those generated by deep learning classification models. We specifically assess the impact of deep learning model architecture, the proportion of label noise in the training data, and the size of the training dataset on three ecological metrics: species richness, occupancy, and activity patterns. Overall, ecological metrics derived from deep neural networks closely match those calculated from expert labels and remain robust to manipulations in the training pipeline. We found that the choice of deep learning model architecture does not impact ecological metrics, and ecological metrics related to the overall community (species richness, community occupancy) were resilient to up to 10% noise in the training dataset and a 50% reduction in the training dataset size. However, we caution that less common species are disproportionately affected by a reduction in deep neural network accuracy, and this has consequences for species-specific metrics (occupancy, diel activity patterns). To ensure the reliability of their findings, practitioners should prioritize creating large, clean training sets with balanced representation across species over exploring numerous deep learning model architectures.

Problem

• Impact of deep learning classification errors on ecological metrics
• Effect of training data quality and quantity on species analysis
• Robustness of ecological metrics to label noise and dataset size

Innovation

• Deep learning automates camera trap image labeling
• Training data size and quality impact ecological metrics
• Model architecture choice does not affect ecological metrics