MedConv: Convolutions Beat Transformers on Long-Tailed Bone Density Prediction

2025-02-02

AI Summary

Transformer-based models for T-score prediction from CT images suffer from high computational overhead and poor suitability for edge deployment, while clinical data exhibiting long-tailed class distributions introduce significant prediction bias. Method: We propose a lightweight 3D convolutional architecture—first empirically validated to outperform Transformers on bone mineral density (BMD) prediction under long-tailed distribution—and introduce a joint Bal-CE (balanced cross-entropy) loss with posterior logit calibration to mitigate class imbalance. Contribution/Results: On the AustinSpine dataset, our method achieves a 21% absolute improvement in accuracy and a 20% gain in ROC AUC over state-of-the-art methods. It also attains 5.3× faster inference speed and reduces GPU memory consumption by 68%, enabling practical deployment on portable clinical devices and resource-constrained edge platforms.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Bone density prediction via CT scans to estimate T-scores is crucial, providing a more precise assessment of bone health compared to traditional methods like X-ray bone density tests, which lack spatial resolution and the ability to detect localized changes. However, CT-based prediction faces two major challenges: the high computational complexity of transformer-based architectures, which limits their deployment in portable and clinical settings, and the imbalanced, long-tailed distribution of real-world hospital data that skews predictions. To address these issues, we introduce MedConv, a convolutional model for bone density prediction that outperforms transformer models with lower computational demands. We also adapt Bal-CE loss and post-hoc logit adjustment to improve class balance. Extensive experiments on our AustinSpine dataset shows that our approach achieves up to 21% improvement in accuracy and 20% in ROC AUC over previous state-of-the-art methods.

Problem

• CT Scan
• Bone Density Prediction
• Transformer Model

Innovation

• MedConv
• Bone Density Prediction
• Data Distribution Challenges