Tree-based Semantic Losses: Application to Sparsely-supervised Large Multi-class Hyperspectral Segmentation

2025-06-26

AI Summary

Hyperspectral medical image segmentation faces challenges including numerous fine-grained tissue classes, extremely sparse annotations, and subtle inter-class semantic distinctions—exacerbated by conventional methods’ neglect of hierarchical semantic relationships among labels, limiting discriminative capability. Method: We propose a semantic-aware loss function grounded in a class taxonomy tree, enabling hierarchical semantic guidance for sparse supervision; integrated with a background-agnostic sparse training framework to jointly optimize in-distribution pixel segmentation accuracy and out-of-distribution pixel detection. Contribution/Results: To our knowledge, this is the first work to realize hierarchical semantic learning on clinically defined 107-class hyperspectral data under sparse supervision. Experiments on large-scale sparsely annotated datasets demonstrate state-of-the-art performance, with significant improvements in distinguishing anatomically subtle tissue structures.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

Hyperspectral imaging (HSI) shows great promise for surgical applications, offering detailed insights into biological tissue differences beyond what the naked eye can perceive. Refined labelling efforts are underway to train vision systems to distinguish large numbers of subtly varying classes. However, commonly used learning methods for biomedical segmentation tasks penalise all errors equivalently and thus fail to exploit any inter-class semantics in the label space. In this work, we introduce two tree-based semantic loss functions which take advantage of a hierarchical organisation of the labels. We further incorporate our losses in a recently proposed approach for training with sparse, background-free annotations. Extensive experiments demonstrate that our proposed method reaches state-of-the-art performance on a sparsely annotated HSI dataset comprising $107$ classes organised in a clinically-defined semantic tree structure. Furthermore, our method enables effective detection of out-of-distribution (OOD) pixels without compromising segmentation performance on in-distribution (ID) pixels.

Problem

• Exploiting inter-class semantics in biomedical segmentation tasks
• Improving sparsely-supervised large multi-class hyperspectral segmentation
• Detecting out-of-distribution pixels without degrading segmentation performance

Innovation

• Tree-based semantic loss functions
• Hierarchical label organization utilization
• Sparse annotation training integration