An automated method of identifying incorrectly labelled images based on the sequences of loss functions of deep learning networks

2026-07-01

AI Summary

This study addresses the critical issue of label noise in medical imaging datasets, where annotation errors affect up to 10% of samples and significantly degrade model performance. The authors propose a method that requires neither additional annotations nor architectural modifications, instead leveraging the temporal dynamics of loss trajectories across multiple training epochs of a deep classification network to automatically flag potentially mislabeled instances. By applying time-series anomaly detection to these loss sequences—a novel approach for identifying annotation noise—the method achieves a recall of 75.31% with only a 4.85% false positive rate on a retinal fundus image dataset. Following expert-in-the-loop verification and correction of flagged labels, model accuracy improves from 95.93% to 96.50%, closely approaching the ideal performance achievable with noise-free data (96.57%), thereby substantially enhancing both data quality and model robustness.

Abstract

Deep learning is widely applied in medical image analysis, but up to 10% of manually labelled images may be incorrect, degrading model performance. This paper proposes an automated method to identify incorrectly labelled medical images by analyzing sequences of loss functions from deep learning classification networks over multiple training epochs. Identified images can be reviewed and relabelled by experts, improving dataset quality and model performance. Two experiments validate the method on a fundus image dataset for referable diabetic retinopathy screening. In the first, 6% (648) of 10,788 gold-standard labels were intentionally flipped. The method identified 75.31% (488) of the flipped samples, with only 4.85% (492) false positives among correctly labelled samples. In the second, reviewing and correcting the 980 identified samples (9.1% of the dataset) and retraining the model improved best accuracy on an independent test set from 95.93% (with 6% label noise) to 96.50% (with 1.5% noise), approaching the ideal 96.57% (with 0% noise). The results demonstrate the method's effectiveness in improving model performance through automated label quality control.

Problem

• label noise
• medical image analysis
• incorrectly labelled images
• deep learning
• dataset quality

Innovation

• label noise detection
• loss trajectory analysis
• medical image quality control
• deep learning
• automated relabeling