Instance Awareness of Multi-class Semantic Segmentation Loss Functions

2026-04-27

AI Summary

This work addresses the challenge of instance and class imbalance in multi-class semantic segmentation, which often leads to the neglect of small lesions and rare classes. The study presents the first effective extension of instance-aware loss functions to multi-class settings by decomposing the problem into one-versus-rest formulations, yielding multi-class versions of the Connected Component (CC) and Blob losses. To further balance gradient contributions across classes and instances, a local inverse-size weighting strategy is introduced at the component level. Evaluated on the BraTS-METS 2025 dataset, the proposed approach demonstrates substantial performance gains: the multi-class CC loss achieves a foreground Dice score of 0.64; the Blob loss attains the best Panoptic Quality (0.40) and Recognition Quality (0.53) at a DSC threshold of 0.5; and inverse-size weighting elevates the Dice score for rare classes to 0.44.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Segmentation
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Instance-sensitive losses for semantic segmentation such as blob loss and CC loss were designed to address instance imbalance, ensuring small lesions generate the same gradient as large ones, but operate only on single-class segmentation. In multi-class settings, class imbalance poses an additional problem: rare classes with few instances receive a disproportionately small share of the training signal. We show that extending instance-sensitive losses to multi-class segmentation via a one-vs-rest class decomposition repurposes them to also address class imbalance, as uniform averaging over classes ensures each class contributes equally regardless of frequency. We further show that inverse-size weighting, which destabilizes training when applied globally due to weight imbalances across rare and common classes, becomes effective when integrated within the per-component loss, confining the reweighting to each component's spatial context. On the BraTS-METS 2025 dataset (260 test cases), multi-class CC loss improves foreground Dice (0.64 +/- 0.26 vs. 0.59 +/- 0.27 baseline) and rare-class Dice, while maintaining Panoptic Quality at DSC threshold 0.5. Multi-class blob loss achieves the best Panoptic Quality at threshold 0.5 (0.40 +/- 0.24 vs. 0.38 +/- 0.25 baseline) and recognition quality (0.53 +/- 0.29 vs. 0.49 +/- 0.30). Integrating inverse-size weighting within the per-component loss increases rare-class Dice to 0.44 +/- 0.36 at the cost of reduced detection quality.

Problem

• instance imbalance
• class imbalance
• multi-class semantic segmentation
• rare classes
• training signal

Innovation

• instance-sensitive loss
• multi-class semantic segmentation
• class imbalance
• inverse-size weighting
• one-vs-rest decomposition