Explaining 3D Computed Tomography Classifiers with Counterfactuals

2025-02-11

AI Summary

To address the challenges of poor interpretability, limited training data, and high GPU memory consumption in deep learning models applied to high-resolution 3D CT imaging, this paper pioneers the adaptation of Latent Shift—a counterfactual generation method—to the 3D medical imaging domain. We propose a “slice encoding–latent-space translation–3D contextual reconstruction” paradigm: lightweight 2D slice encoders extract local features; targeted latent-space translations generate anatomically plausible counterfactuals; and integration of CT-specific preprocessing with anatomical priors ensures 3D-consistent reconstruction. Evaluated on clinical phenotype prediction and lung segmentation tasks, our method produces anatomically valid and semantically interpretable counterfactual instances. It reduces GPU memory usage by 67% and achieves counterfactual fidelity exceeding 92%, effectively alleviating computational and data bottlenecks inherent in high-dimensional 3D medical image analysis.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Counterfactual explanations in medical imaging are critical for understanding the predictions made by deep learning models. We extend the Latent Shift counterfactual generation method from 2D applications to 3D computed tomography (CT) scans. We address the challenges associated with 3D data, such as limited training samples and high memory demands, by implementing a slice-based approach. This method leverages a 2D encoder trained on CT slices, which are subsequently combined to maintain 3D context. We demonstrate this technique on two models for clinical phenotype prediction and lung segmentation. Our approach is both memory-efficient and effective for generating interpretable counterfactuals in high-resolution 3D medical imaging.

Problem

• Extend 2D counterfactual methods to 3D CT
• Address 3D data challenges
• Generate interpretable medical imaging counterfactuals

Innovation

• Extends Latent Shift to 3D CT
• Uses slice-based approach for efficiency
• Combines 2D slices for 3D context