USCNet: Transformer-Based Multimodal Fusion with Segmentation Guidance for Urolithiasis Classification

2026-04-08

AI Summary

Current approaches to renal stone composition analysis rely on postoperative specimens, limiting their utility for preoperative personalized treatment and recurrence prevention. This work proposes USCNet, a novel framework that achieves high-accuracy multimodal classification of kidney stone composition prior to surgery for the first time. Built upon a Transformer architecture, USCNet fuses computed tomography (CT) imaging with electronic health records (EHR), introducing two key innovations: a cross-modal CT-EHR attention mechanism and a segmentation-guided attention module. Furthermore, a dynamic loss function is designed to jointly optimize segmentation and classification tasks. Evaluated on an in-house dataset, USCNet substantially outperforms existing state-of-the-art methods across multiple metrics, demonstrating strong clinical potential for guiding preoperative decision-making and preventive care.

Abstract

Kidney stone disease ranks among the most prevalent conditions in urology, and understanding the composition of these stones is essential for creating personalized treatment plans and preventing recurrence. Current methods for analyzing kidney stones depend on postoperative specimens, which prevents rapid classification before surgery. To overcome this limitation, we introduce a new approach called the Urinary Stone Segmentation and Classification Network (USCNet). This innovative method allows for precise preoperative classification of kidney stones by integrating Computed Tomography (CT) images with clinical data from Electronic Health Records (EHR). USCNet employs a Transformer-based multimodal fusion framework with CT-EHR attention and segmentation-guided attention modules for accurate classification. Moreover, a dynamic loss function is introduced to effectively balance the dual objectives of segmentation and classification. Experiments on an in-house kidney stone dataset show that USCNet demonstrates outstanding performance across all evaluation metrics, with its classification efficacy significantly surpassing existing mainstream methods. This study presents a promising solution for the precise preoperative classification of kidney stones, offering substantial clinical benefits. The source code has been made publicly available: https://github.com/ZhangSongqi0506/KidneyStone.

Problem

• Urolithiasis classification
• Preoperative diagnosis
• Multimodal fusion
• Kidney stone composition
• Clinical decision support

Innovation

• Transformer-based multimodal fusion
• segmentation guidance
• dynamic loss function
• preoperative classification
• kidney stone composition