Semantic-Edge Response Decoding of SAM3 for Zero-Shot Crack Segmentation

2026-07-13

AI Summary

This work addresses the limitations of existing high-precision crack segmentation methods, which rely heavily on pixel-level annotations, and the suboptimal mask quality of general-purpose vision foundation models—such as SAM—when handling thin, low-contrast cracks. The authors propose Semantic Edge Response Decoding (SERD), a novel approach that reveals, for the first time, that the language-conditioned semantic responses within the SAM3 decoder preserve more complete crack evidence than the final output masks. Leveraging this insight, SERD constructs a crack likelihood field and integrates a lightweight edge prior with a global threshold to achieve zero-shot crack segmentation without any fine-tuning or annotated data. Evaluated across six public datasets, the method achieves an average Crack IoU of 61.14%, outperforming SAM3 by 4.63 percentage points and significantly surpassing current zero-shot and open-vocabulary segmentation approaches.

Abstract

Crack segmentation is essential for infrastructure inspection and structural health assessment, but existing high-performance methods typically require task-specific pixel-level annotations and training. Text-promptable vision foundation models enable zero-shot deployment, yet their final mask proposals are poorly suited to thin, fragmented, and low-contrast cracks, whose evidence may be suppressed, truncated, or over-expanded during mask generation. We find that language-conditioned semantic responses within the SAM3 decoder preserve more continuous and complete crack evidence than its final masks. Based on this observation, we propose Semantic-Edge Response Decoding (SERD), which interprets internal responses as a dense crack-likelihood field, calibrates them with a lightweight edge prior, and generates crack masks using a unified global threshold, without annotation or fine-tuning. Experiments on six public datasets show that SERD consistently improves over native SAM3 and outperforms the compared zero-shot and open-vocabulary segmentation methods, achieving an average Crack IoU of 61.14\%, 4.63 points higher than SAM3. Further analyses show that most gains arise from directly decoding internal semantic responses, while edge calibration improves structural recovery and false-positive control without increasing end-to-end inference overhead. These results suggest that, for thin and non-compact targets, internal continuous responses can provide a more transferable interface than the final masks of foundation models. Code is available at: https://github.com/xauat-liushipeng/SERD

Problem

• crack segmentation
• zero-shot learning
• vision foundation models
• thin structures
• mask generation

Innovation

• zero-shot segmentation
• semantic response decoding
• crack segmentation
• foundation model
• edge prior