SegSTRONG-C: Segmenting Surgical Tools Robustly On Non-adversarial Generated Corruptions - An EndoVis'24 Challenge

2024-07-16

arXiv.org

AI Summary

End-to-end deep neural networks for surgical video analysis exhibit severe vulnerability to common non-adversarial clinical degradations—such as bleeding, smoke, and low illumination—hindering real-world clinical deployment. To address this, we introduce SegSTRONG-C, the first benchmark dedicated to evaluating non-adversarial robustness for surgical instrument segmentation. It systematically assesses generalization bottlenecks of existing methods under realistic clinical disturbances and exposes limitations of conventional data augmentation. The challenge advocates a new paradigm for universal degradation robustness, integrating multi-strategy approaches including AutoAugment, robust training, structural regularization, and test-time adaptation. The winning model achieves DSC = 0.9394 and NSD = 0.9301 on the held-out test set—outperforming the strongest baseline by +0.1471 in DSC and +0.2584 in NSD—thereby significantly advancing robustness research in surgical AI.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Surgical data science has seen rapid advancement due to the excellent performance of end-to-end deep neural networks (DNNs) for surgical video analysis. Despite their successes, end-to-end DNNs have been proven susceptible to even minor corruptions, substantially impairing the model's performance. This vulnerability has become a major concern for the translation of cutting-edge technology, especially for high-stakes decision-making in surgical data science. We introduce SegSTRONG-C, a benchmark and challenge in surgical data science dedicated, aiming to better understand model deterioration under unforeseen but plausible non-adversarial corruption and the capabilities of contemporary methods that seek to improve it. Through comprehensive baseline experiments and participating submissions from widespread community engagement, SegSTRONG-C reveals key themes for model failure and identifies promising directions for improving robustness. The performance of challenge winners, achieving an average 0.9394 DSC and 0.9301 NSD across the unreleased test sets with corruption types: bleeding, smoke, and low brightness, shows inspiring improvement of 0.1471 DSC and 0.2584 NSD in average comparing to strongest baseline methods with UNet architecture trained with AutoAugment. In conclusion, the SegSTRONG-C challenge has identified some practical approaches for enhancing model robustness, yet most approaches relied on conventional techniques that have known, and sometimes quite severe, limitations. Looking ahead, we advocate for expanding intellectual diversity and creativity in non-adversarial robustness beyond data augmentation or training scale, calling for new paradigms that enhance universal robustness to corruptions and may enable richer applications in surgical data science.

Problem

• Enhancing DNN robustness to surgical video corruptions
• Addressing model deterioration under non-adversarial corruptions
• Improving segmentation accuracy in corrupted surgical data

Innovation

• Benchmark for non-adversarial corruption robustness
• Comprehensive baseline and community experiments
• Improved metrics with bleeding, smoke, brightness