Imaging foundation model for universal enhancement of non-ideal measurement CT

2024-10-02

arXiv.org

AI Summary

Non-ideal computed tomography (NICT)—characterized by suboptimal acquisition protocols—suffers from degraded image quality and low clinical acceptance, while existing deep learning methods rely heavily on large-scale annotated datasets and exhibit poor generalizability. To address these challenges, we propose TAMP, the first foundation model tailored for NICT. Its core contributions are: (1) a multi-scale integrated Transformer amplifier that jointly incorporates physical priors and data-driven modeling; (2) a physics-informed large-scale synthetic pretraining paradigm, leveraging 10.8 million simulated scans to learn robust representations across diverse protocols, anatomical regions, and noise levels; and (3) parameter-efficient fine-tuning via LoRA-style adaptation requiring only a few slices. Extensive evaluation demonstrates significant PSNR/SSIM improvements across multiple NICT tasks. Clinical validation—including radiologist-blinded assessment and real-world deployment—confirms markedly enhanced diagnostic acceptability, underscoring TAMP’s readiness for clinical translation.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Other Foundations of Computer Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Non-ideal measurement computed tomography (NICT) employs suboptimal imaging protocols to expand CT applications. However, the resulting trade-offs degrade image quality, limiting clinical acceptability. Although deep learning methods have been used to enhance NICT images, their reliance on large training datasets and limited generalizability across diverse settings hinder practical use. We propose the multi-scale integrated Transformer AMPlifier (TAMP), the first imaging foundation model for universal NICT enhancement. Pre-trained on 10.8 million physics-driven simulated NICT images, TAMP generalizes effectively across various NICT settings, defect degrees, and body regions. Moreover, a parameter-efficient fine-tuning strategy enables TAMP to adapt to specific clinical scenarios using only few slices. Extensive experiments, including radiologists and real-world validations, demonstrate that TAMP consistently improves image quality and clinical acceptability, underscoring its significant potential to advance CT imaging and broaden NICT applications in clinical practice.

Problem

• Enhance non-ideal CT image quality
• Improve clinical acceptability of NICT
• Generalize across diverse NICT settings

Innovation

• Transformer-based universal imaging model
• Pre-trained on simulated NICT images
• Parameter-efficient fine-tuning strategy