VerteNet -- A Multi-Context Hybrid CNN Transformer for Accurate Vertebral Landmark Localization in Lateral Spine DXA Images

2025-02-04

AI Summary

This work addresses the low accuracy and scarcity of automated methods for vertebral landmark localization (VLL) in lateral DXA images. We propose a CNN-Transformer hybrid architecture integrating dual-resolution self- and cross-attention (DRSA/DRCA) mechanisms with a multi-context feature fusion block (MCFB). To our knowledge, this is the first method to jointly model multi-scale anatomical structures and long-range spatial dependencies in DXA imagery. It further incorporates automatic ROI cropping and an IVG-guided AAC scoring optimization strategy. Evaluated on 620 multi-vendor clinical DXA scans, the method achieves state-of-the-art performance, significantly improving vertebral localization accuracy, IVG placement consistency, and reliability of abdominal aortic calcification (AAC) quantification. The framework provides a robust, generalizable technical foundation for spinal deformity assessment and non-invasive AAC screening.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Lateral Spine Image (LSI) analysis is important for medical diagnosis, treatment planning, and detailed spinal health assessments. Although modalities like Computed Tomography and Digital X-ray Imaging are commonly used, Dual Energy X-ray Absorptiometry (DXA) is often preferred due to lower radiation exposure, seamless capture, and cost-effectiveness. Accurate Vertebral Landmark Localization (VLL) on LSIs is important to detect spinal conditions like kyphosis and lordosis, as well as assessing Abdominal Aortic Calcification (AAC) using Inter-Vertebral Guides (IVGs). Nonetheless, few automated VLL methodologies have concentrated on DXA LSIs. We present VerteNet, a hybrid CNN-Transformer model featuring a novel dual-resolution attention mechanism in self and cross-attention domains, referred to as Dual Resolution Self-Attention (DRSA) and Dual Resolution Cross-Attention (DRCA). These mechanisms capture the diverse frequencies in DXA images by operating at two different feature map resolutions. Additionally, we design a Multi-Context Feature Fusion Block (MCFB) that efficiently integrates the features using DRSA and DRCA. We train VerteNet on 620 DXA LSIs from various machines and achieve superior results compared to existing methods. We also design an algorithm that utilizes VerteNet's predictions in estimating the Region of Interest (ROI) to detect potential abdominal aorta cropping, where inadequate soft tissue hinders calcification assessment. Additionally, we present a small proof-of-concept study to show that IVGs generated from VLL information can improve inter-reader correlation in AAC scoring, addressing two key areas of disagreement in expert AAC-24 scoring: IVG placement and quality control for full abdominal aorta assessment. The code for this work can be found at https://github.com/zaidilyas89/VerteNet.

Problem

• Accurate vertebral landmark localization in DXA images.
• Detection of spinal conditions and abdominal aortic calcification.
• Improving inter-reader correlation in AAC scoring using IVGs.

Innovation

• Hybrid CNN-Transformer model
• Dual-resolution attention mechanism
• Multi-Context Feature Fusion Block