Rodent-Bench

2026-02-20

AI Summary

This study addresses the limitations of multimodal large language models (MLLMs) in annotating rodent behavioral videos by introducing the first neuroscience-oriented benchmark dataset, encompassing diverse behavioral paradigms and long-duration video sequences. The authors propose a comprehensive evaluation framework incorporating multiple metrics—second-level accuracy, macro F1 score, mean average precision, mutual information, and Matthews correlation coefficient—to systematically assess the performance of state-of-the-art models such as Gemini-2.5-Pro and Qwen-VL-Max. Results reveal significant bottlenecks in temporal segmentation and fine-grained behavior recognition, with models showing only marginal success on a few behaviors like grooming. These findings underscore the current inadequacy of MLLMs in understanding long-form videos and discriminating subtle behavioral patterns critical to neuroscience research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

We present Rodent-Bench, a novel benchmark designed to evaluate the ability of Multimodal Large Language Models (MLLMs) to annotate rodent behaviour footage. We evaluate state-of-the-art MLLMs, including Gemini-2.5-Pro, Gemini-2.5-Flash and Qwen-VL-Max, using this benchmark and find that none of these models perform strongly enough to be used as an assistant for this task. Our benchmark encompasses diverse datasets spanning multiple behavioral paradigms including social interactions, grooming, scratching, and freezing behaviors, with videos ranging from 10 minutes to 35 minutes in length. We provide two benchmark versions to accommodate varying model capabilities and establish standardized evaluation metrics including second-wise accuracy, macro F1, mean average precision, mutual information, and Matthew's correlation coefficient. While some models show modest performance on certain datasets (notably grooming detection), overall results reveal significant challenges in temporal segmentation, handling extended video sequences, and distinguishing subtle behavioral states. Our analysis identifies key limitations in current MLLMs for scientific video annotation and provides insights for future model development. Rodent-Bench serves as a foundation for tracking progress toward reliable automated behavioral annotation in neuroscience research.

Problem

• Multimodal Large Language Models
• rodent behavior annotation
• video understanding
• temporal segmentation
• behavioral neuroscience

Innovation

• Rodent-Bench
• Multimodal Large Language Models
• behavioral video annotation
• temporal segmentation
• standardized evaluation metrics