Structural and Statistical Texture Knowledge Distillation and Learning for Segmentation.

2025-01-30

IEEE Transactions on Pattern Analysis and Machine Intelligence

AI Summary

To address the neglect of low-level texture knowledge—such as boundary sharpness, smoothness, color contrast, and regularity—in semantic segmentation, this paper proposes a Structured and Statistical Texture Knowledge Distillation (SSTKD) framework. Methodologically, it introduces a novel Contourlet Decomposition Module (CDM) to model multi-scale directional contour features, integrates a Texture Intensity Equilibrium Module (TIEM) to capture local-to-global statistical distributions, and designs a Quantization-aware Distribution Loss (QDL) to enhance distillation stability; TIEM is further extended to C-TIEM, and two lightweight, general-purpose segmentation networks—STLNet++ and U-SSNet—are constructed. Extensive experiments across three segmentation tasks and seven mainstream benchmarks demonstrate consistent and significant improvements over state-of-the-art methods, validating that explicit low-level texture modeling substantially enhances both segmentation accuracy and boundary fidelity.

Technology Category

• Computer Vision: Segmentation
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We propose to re-emphasize the low-level texture information in deep networks for semantic segmentation and related knowledge distillation tasks. Low-level texture feature/knowledge is also of vital importance for characterizing the local structural pattern and global statistical property, such as boundary, smoothness, regularity and color contrast, which may not be well addressed by highlevel deep features. In this paper, we are intended to take full advantage of both structural and statistical texture knowledge and propose a novel Structural and Statistical Texture Knowledge Distillation (SSTKD) framework for semantic segmentation. Specifically, Contourlet Decomposition Module (CDM) is introduced to decompose low-level features with iterative Laplacian pyramid and directional filter bank to mine the structural texture knowledge, and Texture Intensity Equalization Module (TIEM) is proposed to extract and enhance statistical texture knowledge with the corresponding Quantization Congruence Loss (QDL). Moreover, we further put forward the Co-occurrence TIEM (C-TIEM) and generic segmentation frameworks, namely STLNet++ and U-SSNet, to enable existing segmentation networks to harvest the structural and statistical texture information more effectively. Extensive experimental results on three segmentation tasks demonstrate the effectiveness of the proposed methods, showing they achieve state-of-the-art performances on seven popular benchmark datasets, respectively.

Problem

• Enhance semantic segmentation using low-level texture features.
• Propose SSTKD framework for structural and statistical texture knowledge.
• Improve segmentation networks with CDM and TIEM modules.

Innovation

• Introduces Structural and Statistical Texture Knowledge Distillation (SSTKD)
• Uses Contourlet Decomposition Module (CDM) for structural texture
• Develops Texture Intensity Equalization Module (TIEM) for statistical texture