MoReMouse: Monocular Reconstruction of Laboratory Mouse

2025-07-06

AI Summary

Addressing the challenge of monocular 3D surface reconstruction for laboratory mice—characterized by textureless surfaces and strong non-rigid deformations—and constrained by the scarcity of real-world training data, this work introduces the first high-fidelity synthetic mouse dataset. We propose a geodesically constrained continuous correspondence embedding method to model anatomical consistency across poses. Our architecture integrates triplane feature representation with a lightweight Transformer-based feed-forward network, coupled with differentiable rendering and a Gaussian-based geometric prior tailored to mouse anatomy. This design significantly improves geometric fidelity and inter-frame stability. Quantitatively, our approach outperforms existing open-source methods across multiple metrics, enabling high-accuracy, robust single-image 3D reconstruction. It further supports high-quality video synthesis and millisecond-level dense surface tracking. The framework establishes a new paradigm for quantitative biomedical analysis of animal behavior.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Laboratory mice play a crucial role in biomedical research, yet accurate 3D mouse surface motion reconstruction remains challenging due to their complex non-rigid geometric deformations and textureless appearance. Moreover, the absence of structured 3D datasets severely hinders the progress beyond sparse keypoint tracking. To narrow the gap, we present MoReMouse, the first monocular dense 3D reconstruction network tailored for laboratory mice. To achieve this goal, we highlight three key designs. First, we construct the first high-fidelity dense-view synthetic dataset for mice, by rendering our self-designed realistic Gaussian mouse avatar. Second, MoReMouse adopts a transformer-based feedforward architecture with triplane representation, achieving high-quality 3D surface generation from a single image. Third, we create geodesic-based continuous correspondence embeddings on mouse surface, which serve as strong semantic priors to improve reconstruction stability and surface consistency. Extensive quantitative and qualitative experiments demonstrate that MoReMouse significantly outperforms existing open-source methods in accuracy and robustness. Video results are available at https://zyyw-eric.github.io/MoreMouse-webpage/.

Problem

• Monocular 3D mouse surface reconstruction from single images
• Overcoming textureless appearance and non-rigid deformation challenges
• Lack of structured 3D datasets for laboratory mice

Innovation

• High-fidelity synthetic dataset using Gaussian mouse avatar
• Transformer-based architecture with triplane representation
• Geodesic-based continuous correspondence embeddings