Learning Neural Parametric 3D Breast Shape Models for Metrical Surface Reconstruction From Monocular RGB Videos

2025-10-15

AI Summary

This work addresses the challenge of low-cost, high-accuracy 3D breast surface reconstruction from monocular RGB video. We propose the first open-source solution that requires no specialized hardware or commercial software. Methodologically, we introduce a local implicit Radiance-Based Signed Distance Field model (liRBSM), which decomposes the global implicit representation into anatomically guided local SDFs—enhancing geometric fidelity, particularly in curvature-sensitive regions. Our end-to-end pipeline integrates off-the-shelf Structure-from-Motion (SfM) with neural radiance field techniques for parametric reconstruction. Experiments demonstrate an average reconstruction error of <2 mm and total processing time under 6 minutes. Compared to the global iRBSM baseline, liRBSM improves accuracy by 37.2% in critical regions including the areola and skin folds. The model, source code, and dataset are fully open-sourced to facilitate preclinical evaluation and personalized medical applications.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Learning with Manifolds
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a neural parametric 3D breast shape model and, based on this model, introduce a low-cost and accessible 3D surface reconstruction pipeline capable of recovering accurate breast geometry from a monocular RGB video. In contrast to widely used, commercially available yet prohibitively expensive 3D breast scanning solutions and existing low-cost alternatives, our method requires neither specialized hardware nor proprietary software and can be used with any device that is able to record RGB videos. The key building blocks of our pipeline are a state-of-the-art, off-the-shelf Structure-from-motion pipeline, paired with a parametric breast model for robust and metrically correct surface reconstruction. Our model, similarly to the recently proposed implicit Regensburg Breast Shape Model (iRBSM), leverages implicit neural representations to model breast shapes. However, unlike the iRBSM, which employs a single global neural signed distance function (SDF), our approach -- inspired by recent state-of-the-art face models -- decomposes the implicit breast domain into multiple smaller regions, each represented by a local neural SDF anchored at anatomical landmark positions. When incorporated into our surface reconstruction pipeline, the proposed model, dubbed liRBSM (short for localized iRBSM), significantly outperforms the iRBSM in terms of reconstruction quality, yielding more detailed surface reconstruction than its global counterpart. Overall, we find that the introduced pipeline is able to recover high-quality 3D breast geometry within an error margin of less than 2 mm. Our method is fast (requires less than six minutes), fully transparent and open-source, and -- together with the model -- publicly available at https://rbsm.re-mic.de/local-implicit.

Problem

• Reconstructing 3D breast geometry from monocular RGB videos
• Overcoming expensive hardware requirements for 3D breast scanning
• Improving reconstruction accuracy using localized neural shape models

Innovation

• Neural parametric 3D breast shape model
• Monocular RGB video reconstruction pipeline
• Local implicit neural representations decomposition