UltraTwin: Towards Cardiac Anatomical Twin Generation from Multi-view 2D Ultrasound

2025-06-29

AI Summary

Current 2D ultrasound lacks the geometric fidelity required for accurate 3D cardiac reconstruction, while clinical 3D ultrasound suffers from limited spatial resolution and practical deployability. To address this, we propose a novel framework for generating high-fidelity, anatomically precise digital twins of the heart from sparse, multi-view 2D ultrasound images. Our key contributions include: (i) introducing the first real-world, strictly co-registered multi-view 2D ultrasound–CT dataset; (ii) designing a topology-aware implicit autoencoder that jointly leverages generative modeling, pseudo-paired learning, multi-view geometric consistency constraints, and implicit neural representations; and (iii) adopting a coarse-to-fine hierarchical reconstruction strategy. Quantitative and qualitative evaluations demonstrate substantial improvements over state-of-the-art baselines in both reconstruction accuracy and structural integrity. The method shows strong potential for clinical translation in personalized cardiac diagnosis and intervention planning.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Echocardiography is routine for cardiac examination. However, 2D ultrasound (US) struggles with accurate metric calculation and direct observation of 3D cardiac structures. Moreover, 3D US is limited by low resolution, small field of view and scarce availability in practice. Constructing the cardiac anatomical twin from 2D images is promising to provide precise treatment planning and clinical quantification. However, it remains challenging due to the rare paired data, complex structures, and US noises. In this study, we introduce a novel generative framework UltraTwin, to obtain cardiac anatomical twin from sparse multi-view 2D US. Our contribution is three-fold. First, pioneered the construction of a real-world and high-quality dataset containing strictly paired multi-view 2D US and CT, and pseudo-paired data. Second, we propose a coarse-to-fine scheme to achieve hierarchical reconstruction optimization. Last, we introduce an implicit autoencoder for topology-aware constraints. Extensive experiments show that UltraTwin reconstructs high-quality anatomical twins versus strong competitors. We believe it advances anatomical twin modeling for potential applications in personalized cardiac care.

Problem

• Generate 3D cardiac anatomical twin from 2D ultrasound
• Overcome limitations of low-resolution 3D ultrasound
• Address challenges of rare paired data and noise

Innovation

• Generative framework for cardiac twin from 2D US
• Coarse-to-fine hierarchical reconstruction optimization
• Implicit autoencoder with topology-aware constraints