Mouse Lockbox Dataset: Behavior Recognition for Mice Solving Lockboxes

2025-05-21

AI Summary

This study addresses the challenge of fine-grained operant behavior recognition in computational neuroscience. We propose the first multi-view video analysis framework specifically designed for automated recognition of mouse manipulative behaviors during complex mechanical lock-box puzzle solving. Methodologically, we construct a synchronized triple-view video dataset exceeding 110 hours—featuring full-frame manual annotations for two mice (13% of total data)—and introduce, for the first time, a naturalistic behavioral paradigm involving active manipulation of high-difficulty mechanical devices by mice. Our approach integrates pose-estimation-driven keypoint tracking with a multi-view fusion action classification model. Key contributions include: (1) releasing the first benchmark dataset for lock-box puzzle-solving behavior (DOI: 10.14279/depositonce-23850); (2) substantially improving accuracy and generalizability in manipulative action recognition, surpassing limitations of prior simple or social behavior analyses; and (3) establishing a reproducible, high spatiotemporal-resolution behavioral quantification paradigm to support neural mechanism investigation.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Intelligent Robots: Manipulation
• Machine Learning: Multi-instance/Multi-view Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Machine learning and computer vision methods have a major impact on the study of natural animal behavior, as they enable the (semi-)automatic analysis of vast amounts of video data. Mice are the standard mammalian model system in most research fields, but the datasets available today to refine such methods focus either on simple or social behaviors. In this work, we present a video dataset of individual mice solving complex mechanical puzzles, so-called lockboxes. The more than 110 hours of total playtime show their behavior recorded from three different perspectives. As a benchmark for frame-level action classification methods, we provide human-annotated labels for all videos of two different mice, that equal 13% of our dataset. Our keypoint (pose) tracking-based action classification framework illustrates the challenges of automated labeling of fine-grained behaviors, such as the manipulation of objects. We hope that our work will help accelerate the advancement of automated action and behavior classification in the computational neuroscience community. Our dataset is publicly available at https://doi.org/10.14279/depositonce-23850

Problem

• Develop a dataset for mice solving complex mechanical puzzles
• Benchmark frame-level action classification methods
• Advance automated behavior classification in neuroscience

Innovation

• Uses machine learning for behavior analysis
• Tracks keypoints for action classification
• Provides multi-perspective video dataset