XAttn-BMD: Multimodal Deep Learning with Cross-Attention for Femoral Neck Bone Mineral Density Estimation

2025-11-18

AI Summary

This study addresses the challenge of accurately predicting femoral neck bone mineral density (BMD) for early osteoporosis screening. We propose a multimodal deep learning framework that jointly leverages hip X-ray images and structured clinical metadata. Key contributions include: (1) a bidirectional cross-attention mechanism enabling dynamic, mutually reinforcing fusion of imaging and clinical features; and (2) a weighted smooth L1 loss function designed to enhance regression sensitivity near clinically critical BMD thresholds (e.g., diagnostic cutoffs). Evaluated on the Hertfordshire cohort, our model achieves a 16.7% reduction in mean squared error (MSE), a 6.03% reduction in mean absolute error (MAE), and a 16.4% improvement in coefficient of determination (R²) over baseline methods. Furthermore, it demonstrates strong clinical utility in binary osteoporosis risk classification. The approach significantly improves both the accuracy and practicality of noninvasive BMD estimation.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Poor bone health is a significant public health concern, and low bone mineral density (BMD) leads to an increased fracture risk, a key feature of osteoporosis. We present XAttn-BMD (Cross-Attention BMD), a multimodal deep learning framework that predicts femoral neck BMD from hip X-ray images and structured clinical metadata. It utilizes a novel bidirectional cross-attention mechanism to dynamically integrate image and metadata features for cross-modal mutual reinforcement. A Weighted Smooth L1 loss is tailored to address BMD imbalance and prioritize clinically significant cases. Extensive experiments on the data from the Hertfordshire Cohort Study show that our model outperforms the baseline models in regression generalization and robustness. Ablation studies confirm the effectiveness of both cross-attention fusion and the customized loss function. Experimental results show that the integration of multimodal data via cross-attention outperforms naive feature concatenation without cross-attention, reducing MSE by 16.7%, MAE by 6.03%, and increasing the R2 score by 16.4%, highlighting the effectiveness of the approach for femoral neck BMD estimation. Furthermore, screening performance was evaluated using binary classification at clinically relevant femoral neck BMD thresholds, demonstrating the model's potential in real-world scenarios.

Problem

• Estimating femoral neck bone mineral density using multimodal data
• Integrating X-ray images and clinical metadata via cross-attention
• Addressing BMD imbalance with a customized weighted loss function

Innovation

• Multimodal deep learning with cross-attention mechanism
• Bidirectional cross-attention integrates image and metadata features
• Weighted Smooth L1 loss addresses BMD imbalance