Fast Multi-Stack Slice-to-Volume Reconstruction via Multi-Scale Unrolled Optimization

2026-01-12

AI Summary

This work addresses the challenging slice-to-volume reconstruction problem of jointly reconstructing 3D anatomical structures from multiple misaligned 2D stacks while simultaneously estimating their poses. The authors propose a novel approach that integrates a multi-scale fully convolutional network with a lightweight unrolled optimization scheme. By learning non-rigid displacement fields in an end-to-end manner to align multi-stack slices, this method is the first to combine multi-scale unrolled optimization with deep learning for slice-to-volume reconstruction. Evaluated on fetal brain MRI, it achieves slice registration in under one second and high-quality 3D reconstruction within ten seconds, matching the accuracy of state-of-the-art iterative methods while offering substantially improved speed. The framework enables real-time scanning feedback and generalizes effectively to other applications such as whole-fetus and placental MRI.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Fully convolutional networks have become the backbone of modern medical imaging due to their ability to learn multi-scale representations and perform end-to-end inference. Yet their potential for slice-to-volume reconstruction (SVR), the task of jointly estimating 3D anatomy and slice poses from misaligned 2D acquisitions, remains underexplored. We introduce a fast convolutional framework that fuses multiple orthogonal 2D slice stacks to recover coherent 3D structure and refines slice alignment through lightweight model-based optimization. Applied to fetal brain MRI, our approach reconstructs high-quality 3D volumes in under 10s, with 1s slice registration and accuracy on par with state-of-the-art iterative SVR pipelines, offering more than speedup. The framework uses non-rigid displacement fields to represent transformations, generalizing to other SVR problems like fetal body and placental MRI. Additionally, the fast inference time paves the way for real-time, scanner-side volumetric feedback during MRI acquisition.

Problem

• slice-to-volume reconstruction
• 3D anatomy estimation
• slice pose alignment
• fetal MRI
• multi-stack fusion

Innovation

• slice-to-volume reconstruction
• multi-scale unrolled optimization
• fully convolutional networks
• non-rigid registration
• real-time MRI