WaveSeg: Enhancing Segmentation Precision via High-Frequency Prior and Mamba-Driven Spectrum Decomposition

2025-10-23

AI Summary

Existing semantic segmentation methods often suffer from insufficient decoder capacity, struggling to jointly model contextual information and preserve boundary details. To address this, we propose WaveSeg, the first framework that jointly refines features in both spatial and wavelet domains. WaveSeg explicitly incorporates high-frequency priors to enhance edge responsiveness; introduces a spectral decomposition attention module that synergistically integrates wavelet high-frequency components with Mamba’s long-range modeling capability; employs reparameterized convolutions to ensure low-frequency semantic integrity; and adopts residual-guided multi-scale fusion to improve feature consistency. Evaluated on multiple standard benchmarks, WaveSeg consistently outperforms state-of-the-art methods, achieving superior trade-offs among mean Intersection-over-Union (mIoU), boundary accuracy, and inference efficiency. These results empirically validate the effectiveness of spectrum-aware modeling for fine-grained semantic segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multimodal Learning
• Natural Language Processing: Lexical Semantics and Morphology

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

While recent semantic segmentation networks heavily rely on powerful pretrained encoders, most employ simplistic decoders, leading to suboptimal trade-offs between semantic context and fine-grained detail preservation. To address this, we propose a novel decoder architecture, WaveSeg, which jointly optimizes feature refinement in spatial and wavelet domains. Specifically, high-frequency components are first learned from input images as explicit priors to reinforce boundary details at early stages. A multi-scale fusion mechanism, Dual Domain Operation (DDO), is then applied, and the novel Spectrum Decomposition Attention (SDA) block is proposed, which is developed to leverage Mamba's linear-complexity long-range modeling to enhance high-frequency structural details. Meanwhile, reparameterized convolutions are applied to preserve low-frequency semantic integrity in the wavelet domain. Finally, a residual-guided fusion integrates multi-scale features with boundary-aware representations at native resolution, producing semantically and structurally rich feature maps. Extensive experiments on standard benchmarks demonstrate that WaveSeg, leveraging wavelet-domain frequency prior with Mamba-based attention, consistently outperforms state-of-the-art approaches both quantitatively and qualitatively, achieving efficient and precise segmentation.

Problem

• Enhancing segmentation precision with high-frequency priors
• Optimizing feature refinement in spatial and wavelet domains
• Improving boundary details and semantic context preservation

Innovation

• Wavelet-domain joint optimization for feature refinement
• Mamba-based linear-complexity attention for spectrum decomposition
• Multi-scale fusion with boundary-aware representations