Cross-Modal Guidance for Fast Diffusion-Based Computed Tomography

2026-03-01

AI Summary

This work addresses the challenge of poor image quality in sparse-view neutron computed tomography (CT) reconstruction due to limited data availability. While existing cross-modal approaches rely on retraining diffusion models—an expensive and impractical requirement—this study proposes a novel cross-modal guidance mechanism that leverages readily accessible auxiliary modalities, such as X-ray CT, without necessitating retraining of the diffusion prior. The method achieves, for the first time, retraining-free fusion of multimodal information, significantly enhancing both reconstruction quality and speed while reducing dependence on costly neutron data. Furthermore, it demonstrates robustness even when the auxiliary modality is imperfect. Experimental results confirm that incorporating X-ray CT from the same specimen effectively improves neutron CT reconstruction, highlighting the approach’s practicality and generalizability.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Diffusion models have emerged as powerful priors for solving inverse problems in computed tomography (CT). In certain applications, such as neutron CT, it can be expensive to collect large amounts of measurements even for a single scan, leading to sparse data sets from which it is challenging to obtain high quality reconstructions even with diffusion models. One strategy to mitigate this challenge is to leverage a complementary, easily available imaging modality; however, such approaches typically require retraining the diffusion model with large datasets. In this work, we propose incorporating an additional modality without retraining the diffusion prior, enabling accelerated imaging of costly modalities. We further examine the impact of imperfect side modalities on cross-modal guidance. Our method is evaluated on sparse-view neutron computed tomography, where reconstruction quality is substantially improved by incorporating X-ray computed tomography of the same samples.

Problem

• cross-modal guidance
• diffusion models
• computed tomography
• sparse-view reconstruction
• neutron CT

Innovation

• cross-modal guidance
• diffusion models
• sparse-view CT
• neutron computed tomography
• inverse problems