How Well Do LLMs Simulate Survey Responses Following a Breast Cancer Screening Intervention?
研究使用大型语言模型模拟乳腺癌筛查干预前后的调查响应分布,通过与真实数据对比评估其准确性,发现基于个人资料的代理比零样本预测更准确。
研究使用大型语言模型模拟乳腺癌筛查干预前后的调查响应分布,通过与真实数据对比评估其准确性,发现基于个人资料的代理比零样本预测更准确。
This study challenges the common assumption that explanation stability is an inherent property of deep learning models, demonstrating instead that it emerges from the interaction between the model and the attribution method. Systematically evaluating DenseNet201, ResNet50V2, and InceptionV3 on chest X-ray data using multiple attribution techniques—including LayerCAM and GradCAM++—the authors employ IoU and AUC (both exceeding 99%) as stability metrics. Their results reveal that model stability rankings vary dramatically across attribution methods: for instance, LayerCAM identifies InceptionV3 as the most stable model (IoU = 0.777), whereas GradCAM++ reduces its stability by 17.3%. These findings underscore that attribution method choice critically influences stability assessments and highlight the necessity of cross-method validation to avoid biased conclusions.
This study addresses a critical gap in the evaluation of explainability for medical image classification models, which has predominantly emphasized localization accuracy while overlooking whether models employ consistent spatial reasoning strategies across pathologically similar samples. To bridge this gap, we introduce C-Score (Consistency Score), a novel, annotation-free metric that quantifies intra-class consistency of Class Activation Map (CAM) explanations using confidence-weighted soft Intersection-over-Union with intensity emphasis. Through transfer learning experiments on the Kermany chest X-ray dataset—combining six CAM variants (including Grad-CAM) with DenseNet201, InceptionV3, and ResNet50V2—we uncover three distinct mechanisms by which explanation consistency decouples from AUC performance. Notably, C-Score detects ScoreCAM degradation one checkpoint before a significant AUC drop, offering an early, explanation-quality-based warning signal to inform clinical model selection and deployment.
This study addresses the trade-off between accuracy and interpretability in chest X-ray multi-class classification, where fine-tuning improves performance but may induce semantic drift in the visual evidence underpinning model explanations, thereby undermining clinical trust. The authors propose a two-stage training protocol and systematically compare attribution maps generated under transfer learning versus full fine-tuning across DenseNet201, ResNet50V2, and InceptionV3. They demonstrate for the first time that explanation stability is jointly determined by model architecture, optimization phase, and attribution method—with stability rankings even reversing across different attribution techniques. Using LayerCAM and GradCAM++ alongside no-reference metrics such as IoU to assess spatial consistency, they find that coarse-grained anatomical localization remains stable, whereas fine-grained evidential structures are highly architecture-dependent, revealing that high accuracy does not guarantee reliable interpretability.
研究使用大型语言模型模拟乳腺癌筛查干预前后的调查响应分布,通过与真实数据对比评估其准确性,发现基于个人资料的代理比零样本预测更准确。
This study challenges the common assumption that explanation stability is an inherent property of deep learning models, demonstrating instead that it emerges from the interaction between the model and the attribution method. Systematically evaluating DenseNet201, ResNet50V2, and InceptionV3 on chest X-ray data using multiple attribution techniques—including LayerCAM and GradCAM++—the authors employ IoU and AUC (both exceeding 99%) as stability metrics. Their results reveal that model stability rankings vary dramatically across attribution methods: for instance, LayerCAM identifies InceptionV3 as the most stable model (IoU = 0.777), whereas GradCAM++ reduces its stability by 17.3%. These findings underscore that attribution method choice critically influences stability assessments and highlight the necessity of cross-method validation to avoid biased conclusions.
This study addresses a critical gap in the evaluation of explainability for medical image classification models, which has predominantly emphasized localization accuracy while overlooking whether models employ consistent spatial reasoning strategies across pathologically similar samples. To bridge this gap, we introduce C-Score (Consistency Score), a novel, annotation-free metric that quantifies intra-class consistency of Class Activation Map (CAM) explanations using confidence-weighted soft Intersection-over-Union with intensity emphasis. Through transfer learning experiments on the Kermany chest X-ray dataset—combining six CAM variants (including Grad-CAM) with DenseNet201, InceptionV3, and ResNet50V2—we uncover three distinct mechanisms by which explanation consistency decouples from AUC performance. Notably, C-Score detects ScoreCAM degradation one checkpoint before a significant AUC drop, offering an early, explanation-quality-based warning signal to inform clinical model selection and deployment.
This study addresses the trade-off between accuracy and interpretability in chest X-ray multi-class classification, where fine-tuning improves performance but may induce semantic drift in the visual evidence underpinning model explanations, thereby undermining clinical trust. The authors propose a two-stage training protocol and systematically compare attribution maps generated under transfer learning versus full fine-tuning across DenseNet201, ResNet50V2, and InceptionV3. They demonstrate for the first time that explanation stability is jointly determined by model architecture, optimization phase, and attribution method—with stability rankings even reversing across different attribution techniques. Using LayerCAM and GradCAM++ alongside no-reference metrics such as IoU to assess spatial consistency, they find that coarse-grained anatomical localization remains stable, whereas fine-grained evidential structures are highly architecture-dependent, revealing that high accuracy does not guarantee reliable interpretability.