Context Determines Optimal Architecture in Materials Segmentation

2026-02-04

AI Summary

This study addresses the lack of guidance in selecting segmentation architectures tailored to diverse imaging modalities in materials science, which hampers model generalization. We present the first cross-modal evaluation framework that systematically assesses six prominent encoder-decoder architectures—including UNet and DeepLabv3+—across seven materials datasets spanning scanning electron microscopy (SEM), atomic force microscopy (AFM), X-ray computed tomography (XCT), and optical microscopy. By integrating out-of-distribution detection with counterfactual explanations, our approach provides actionable reliability feedback. Our analysis reveals, for the first time, systematic variations in segmentation performance driven by imaging context, and we propose an integrated solution that combines architecture recommendation, trustworthy evaluation, and interpretability. This framework substantially enhances the reliability of model deployment in novel materials characterization scenarios.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Deep Neural Architectures and Foundation Models
• Cognitive Modeling & Cognitive Systems: (Computational) Cognitive Architectures

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

Segmentation architectures are typically benchmarked on single imaging modalities, obscuring deployment-relevant performance variations: an architecture optimal for one modality may underperform on another. We present a cross-modal evaluation framework for materials image segmentation spanning SEM, AFM, XCT, and optical microscopy. Our evaluation of six encoder-decoder combinations across seven datasets reveals that optimal architectures vary systematically by context: UNet excels for high-contrast 2D imaging while DeepLabv3+ is preferred for the hardest cases. The framework also provides deployment feedback via out-of-distribution detection and counterfactual explanations that reveal which microstructural features drive predictions. Together, the architecture guidance, reliability signals, and interpretability tools address a practical gap in materials characterization, where researchers lack tools to select architectures for their specific imaging setup or assess when models can be trusted on new samples.

Problem

• materials segmentation
• imaging modality
• architecture selection
• model reliability
• cross-modal evaluation

Innovation

• cross-modal evaluation
• materials segmentation
• out-of-distribution detection
• counterfactual explanations
• architecture selection