Temporally Consistent and Controllable Video Generation of 2D Cine CMR via Latent Space Motion Modeling

2026-06-08

AI Summary

This work addresses the scarcity of publicly available cardiac cine magnetic resonance (Cine CMR) data, which hinders the development of data-driven models, by proposing a novel text-to-video generation framework that explicitly disentangles and models cardiac spatial anatomy and temporal dynamics in latent space for the first time. The method first fine-tunes a diffusion model to generate anatomically plausible initial frames from textual prompts, then employs a cardiac phase–aware latent optical flow model to synthesize temporally coherent sequences, ensuring both anatomical fidelity and motion consistency across the cardiac cycle. Experimental results demonstrate that the generated Cine CMR videos achieve state-of-the-art performance with a Fréchet Inception Distance (FID) of 31.68 and a CLIP score of 31.04, significantly advancing the quality of medical video synthesis by enabling text-controllable, physiologically plausible, and high-fidelity Cine CMR generation.

Abstract

Cine cardiac magnetic resonance is the gold standard for assessing cardiac function, but the scarcity of public datasets limits the development of advanced data-driven models. To address this limitation, we propose a generative method for synthesizing temporally coherent and anatomically consistent cardiac sequences. Our text-to-video framework decouples cardiac spatial structure from temporal motion. First, a fine-tuned diffusion model synthesizes an initial frame from a clinical text prompt, controlling anatomical features. Then, a latent flow model conditioned on a cardiac phase embedding generates the complete cardiac motion, ensuring spatial consistency and temporal control. Our model generates anatomically and pathologically diverse sequences with high temporal coherence and strong fidelity to input prompts, achieving a FID of 31.68 for image realism and a CLIP score of 31.04 for text-image alignment. These experimental results highlight its potential to produce high-fidelity, on-demand medical data, offering a scalable solution to data scarcity.

Problem

• Cine CMR
• data scarcity
• cardiac function
• medical video generation
• temporal consistency

Innovation

• latent space motion modeling
• temporally consistent video generation
• text-to-video diffusion
• cardiac phase embedding
• anatomical consistency