MTT-Bench: Predicting Social Dominance in Mice via Multimodal Large Language Models

2026-04-24

AI Summary

This work proposes a novel approach to automatically predict social dominance hierarchies in mice directly from raw interaction videos, without requiring manual annotations or domain-specific models. By introducing the MTT-Bench benchmark dataset and leveraging multimodal large language models (MLLMs)—applied for the first time in ethology—the method enables end-to-end, zero-shot inference of social rank. The model, built upon existing multimodal architectures with minimal fine-tuning and trained solely on Tube Test labeling data, accurately predicts dominance relationships even in the absence of test-time labels. Experimental results demonstrate a high degree of agreement between the model’s predictions and established Tube Test rankings, confirming the effectiveness and generalization capability of general-purpose multimodal foundation models in animal behavior analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Understanding social dominance in animal behavior is critical for neuroscience and behavioral studies. In this work, we explore the capability of Multimodal Large Language Models(MLLMs) to analyze raw behavioral video of mice and predict their dominance hierarchy. We introduce MTT-Bench, a novel benchmark comprising annotated videos of pairwise mouse interactions for Mouse Tube Test analysis. Building on existing MLLM architectures, we fine-tune these models to perform zero-shot inference on unseen behavioral sequences, predicting social dominance without explicit labels during testing. Our framework demonstrates promising results, showing high agreement with tube test rankings. This work opens a new direction for applying foundation models to ethology and social behavior analysis, without the need to design domain-specific models.

Problem

• social dominance
• mouse behavior
• multimodal large language models
• ethology
• dominance hierarchy

Innovation

• Multimodal Large Language Models
• MTT-Bench
• social dominance prediction
• zero-shot inference
• ethology