Automated Radiographic Total Sharp Score (ARTSS) in Rheumatoid Arthritis: A Solution to Reduce Inter-Intra Reader Variation and Enhancing Clinical Practice

2025-09-08

AI Summary

Rheumatoid arthritis (RA) assessment using the Total Sharp/Van der Heijde Score (TSS) relies on manual radiographic interpretation, suffering from high time consumption, strong subjectivity, and substantial inter- and intra-observer variability. To address this, we propose the first end-to-end automated TSS estimation framework for full-hand radiographs, uniquely tackling two key clinical challenges: joint disappearance and variable joint count. Our method integrates ResNet50-based image correction, U-Net³ for hand segmentation, YOLOv7 for joint localization, and a multi-model ensemble—including VGG16, ResNet50, DenseNet201, EfficientNetB0, and Vision Transformer (ViT)—for TSS regression. ViT achieves optimal performance with a Huber loss of 0.87. Experimental results demonstrate 99% joint detection accuracy, robust generalization on external test sets, and significant reduction in human-induced variability—thereby enhancing both assessment efficiency and inter-rater consistency.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Manipulation
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Assessing the severity of rheumatoid arthritis (RA) using the Total Sharp/Van Der Heijde Score (TSS) is crucial, but manual scoring is often time-consuming and subjective. This study introduces an Automated Radiographic Sharp Scoring (ARTSS) framework that leverages deep learning to analyze full-hand X-ray images, aiming to reduce inter- and intra-observer variability. The research uniquely accommodates patients with joint disappearance and variable-length image sequences. We developed ARTSS using data from 970 patients, structured into four stages: I) Image pre-processing and re-orientation using ResNet50, II) Hand segmentation using UNet.3, III) Joint identification using YOLOv7, and IV) TSS prediction using models such as VGG16, VGG19, ResNet50, DenseNet201, EfficientNetB0, and Vision Transformer (ViT). We evaluated model performance with Intersection over Union (IoU), Mean Average Precision (MAP), mean absolute error (MAE), Root Mean Squared Error (RMSE), and Huber loss. The average TSS from two radiologists was used as the ground truth. Model training employed 3-fold cross-validation, with each fold consisting of 452 training and 227 validation samples, and external testing included 291 unseen subjects. Our joint identification model achieved 99% accuracy. The best-performing model, ViT, achieved a notably low Huber loss of 0.87 for TSS prediction. Our results demonstrate the potential of deep learning to automate RA scoring, which can significantly enhance clinical practice. Our approach addresses the challenge of joint disappearance and variable joint numbers, offers timesaving benefits, reduces inter- and intra-reader variability, improves radiologist accuracy, and aids rheumatologists in making more informed decisions.

Problem

• Reducing inter-intra reader variability in rheumatoid arthritis radiographic scoring
• Automating time-consuming manual TSS assessment using deep learning
• Addressing joint disappearance and variable-length sequences in RA imaging

Innovation

• Deep learning framework automates RA radiographic scoring
• Handles joint disappearance and variable-length image sequences
• Vision Transformer achieves lowest Huber loss for prediction