MeshMetrics: A Precise Implementation of Distance-Based Image Segmentation Metrics

2025-09-06

AI Summary

Image segmentation evaluation suffers from a reproducibility crisis, primarily due to inconsistent implementations of distance-based metrics—such as the Hausdorff Distance (HD) and Normalized Surface Distance (NSD)—across open-source toolkits, compounded by significant discretization errors inherent in voxel-based approximations. To address this, we propose the first mesh-based high-precision distance computation framework, which replaces conventional grid-based surface approximations with exact triangular mesh reconstruction and rigorous geometric distance solving, thereby eliminating discretization bias at its source. Theoretical analysis and numerical experiments demonstrate substantial improvements in metric reliability: absolute HD error decreases by over 100 mm, and relative NSD error is reduced by 30%. Our implementation is released as an open-source Python package, establishing a new, reproducible, and high-fidelity benchmark for segmentation evaluation.

Technology Category

• Computer Vision: Segmentation
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

The surge of research in image segmentation has yielded remarkable performance gains but also exposed a reproducibility crisis. A major contributor is performance evaluation, where both selection and implementation of metrics play critical roles. While recent efforts have improved the former, the reliability of metric implementation has received far less attention. Pitfalls in distance-based metric implementation can lead to considerable discrepancies between common open-source tools, for instance, exceeding 100 mm for the Hausdorff distance and 30%pt for the normalized surface distance for the same pair of segmentations. To address these pitfalls, we introduce MeshMetrics, a mesh-based framework that provides a more precise computation of distance-based metrics than conventional grid-based approaches. Through theoretical analysis and empirical validation, we demonstrate that MeshMetrics achieves higher accuracy and precision than established tools, and is substantially less affected by discretization artifacts, such as distance quantization. We release MeshMetrics as an open-source Python package, available at https://github.com/gasperpodobnik/MeshMetrics.

Problem

• Addressing reproducibility crisis in image segmentation evaluation
• Improving reliability of distance-based metric implementations
• Reducing discretization artifacts in segmentation performance assessment

Innovation

• Mesh-based framework for precise metric computation
• Higher accuracy and precision than grid-based tools
• Reduced discretization artifacts in distance metrics