TerraIncognita: A Dynamic Benchmark for Species Discovery Using Frontier Models

2025-05-29

AI Summary

Insect species discovery remains heavily manual, inefficient, and inadequate for addressing the biodiversity crisis. Method: We propose an open-world unknown-species recognition framework powered by multimodal AI, introducing TerraIncognita—a dynamic benchmark enabling continual evaluation of undescribed (unknown) species. It integrates known/unknown images, simulates realistic ecological distributions, and establishes a novel evaluation paradigm combining hierarchical classification, out-of-distribution (OOD) rejection, and expert-aligned interpretability. Our approach unifies multimodal large models, hierarchical architectures, dynamic data management, and interpretable generation modules. Contribution/Results: Experiments reveal strong performance at the order level (F1 > 90%), but critically low accuracy at the species level (F1 < 2%), highlighting a fundamental fine-grained discovery bottleneck. TerraIncognita supports quarterly benchmark expansion and longitudinal model tracking, establishing an evolvable AI evaluation infrastructure for scalable biodiversity monitoring.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The rapid global loss of biodiversity, particularly among insects, represents an urgent ecological crisis. Current methods for insect species discovery are manual, slow, and severely constrained by taxonomic expertise, hindering timely conservation actions. We introduce TerraIncognita, a dynamic benchmark designed to evaluate state-of-the-art multimodal models for the challenging problem of identifying unknown, potentially undescribed insect species from image data. Our benchmark dataset combines a mix of expertly annotated images of insect species likely known to frontier AI models, and images of rare and poorly known species, for which few/no publicly available images exist. These images were collected from underexplored biodiversity hotspots, realistically mimicking open-world discovery scenarios faced by ecologists. The benchmark assesses models' proficiency in hierarchical taxonomic classification, their capability to detect and abstain from out-of-distribution (OOD) samples representing novel species, and their ability to generate explanations aligned with expert taxonomic knowledge. Notably, top-performing models achieve over 90% F1 at the Order level on known species, but drop below 2% at the Species level, highlighting the sharp difficulty gradient from coarse to fine taxonomic prediction (Order $ ightarrow$ Family $ ightarrow$ Genus $ ightarrow$ Species). TerraIncognita will be updated regularly, and by committing to quarterly dataset expansions (of both known and novel species), will provide an evolving platform for longitudinal benchmarking of frontier AI methods. All TerraIncognita data, results, and future updates are available href{https://baskargroup.github.io/TerraIncognita/}{here}.

Problem

• Evaluating AI models for identifying unknown insect species from images
• Assessing model performance in hierarchical taxonomic classification
• Detecting novel species and generating expert-aligned explanations

Innovation

• Dynamic benchmark for unknown insect species discovery
• Multimodal models evaluate hierarchical taxonomic classification
• Regular dataset expansions for evolving AI benchmarking