IMITATE: Image Registration with Context for Unknown Time Frame Recovery

2025-04-14

IEEE International Symposium on Biomedical Imaging

AI Summary

This work addresses the challenges of phase-image registration and reconstruction artifacts in respiratory-gated 4D-CT—arising from irregular breathing patterns, respiratory delay, and decoupling between external and internal motion—by proposing a novel reference-free conditional registration paradigm for unknown respiratory phases. Methodologically, we introduce the first conditional U-Net architecture explicitly encoding clinical priors such as respiratory amplitude, without requiring a fixed reference image; it integrates context-aware deformation estimation, 4D temporal modeling, and real-time voxel-wise motion interpolation. Evaluated on real clinical data, our approach achieves end-to-end, millisecond-latency, artifact-free 3D volume reconstruction, significantly suppressing stitching artifacts. To our knowledge, this is the first systematic integration of conditional generative modeling into radiotherapy image registration, establishing a new pathway for precise tumor tracking under irregular respiration.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Multi-modal Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: User modeling for targeted and personalized online advertising
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this paper, we formulate a novel image registration formalism dedicated to the estimation of unknown condition-related images, based on two or more known images and their associated conditions. We show how to practically model this formalism by using a new conditional U-Net architecture, which fully takes into account the conditional information and does not need any fixed image. Our formalism is then applied to image moving tumors for radiotherapy treatment at different breathing amplitude using 4D-CT (3D+t) scans in thoracoab-dominal regions. This driving application is particularly complex as it requires to stitch a collection of sequential 2D slices into several 3D volumes at different organ positions. Movement interpolation with standard methods then generates well known reconstruction artefacts in the assembled volumes due to irregular patient breathing, hysteresis and poor correlation of breathing signal to internal motion. Results obtained on 4D-CT clinical data showcase artefact-free volumes achieved through real-time latencies.

Problem

• Estimating unknown condition-related images from known images
• Modeling image registration with conditional U-Net architecture
• Reducing reconstruction artefacts in 4D-CT scans for radiotherapy

Innovation

• Conditional U-Net for image registration
• Handles unknown condition-related images
• Artefact-free 4D-CT volume reconstruction