Power of Boundary and Reflection: Semantic Transparent Object Segmentation using Pyramid Vision Transformer with Transparent Cues

2025-12-07

AI Summary

Transparent objects—such as glass—pose significant challenges for semantic segmentation due to their textureless surfaces and low contrast, making boundary delineation and reflection modeling particularly difficult; existing methods struggle to capture these intrinsic optical properties effectively. To address this, we propose TransCues, the first framework that explicitly fuses boundary cues and reflection cues in a complementary manner within a pyramid Vision Transformer encoder-decoder architecture, forming a transparent-cue enhancement module. This design jointly encodes geometric contour priors and optical reflection priors of transparent materials, thereby substantially improving texture-aware perception. Evaluated on benchmark datasets—including Trans10K-v2, MSD, and RGBD-Mirror—TransCues achieves up to a 13.1% improvement in mean Intersection-over-Union (mIoU) over state-of-the-art methods. These results comprehensively validate the effectiveness and generalizability of synergistic boundary-reflection modeling for transparent object segmentation.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Security and Privacy: Data transparency and provenance
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

Glass is a prevalent material among solid objects in everyday life, yet segmentation methods struggle to distinguish it from opaque materials due to its transparency and reflection. While it is known that human perception relies on boundary and reflective-object features to distinguish glass objects, the existing literature has not yet sufficiently captured both properties when handling transparent objects. Hence, we propose incorporating both of these powerful visual cues via the Boundary Feature Enhancement and Reflection Feature Enhancement modules in a mutually beneficial way. Our proposed framework, TransCues, is a pyramidal transformer encoder-decoder architecture to segment transparent objects. We empirically show that these two modules can be used together effectively, improving overall performance across various benchmark datasets, including glass object semantic segmentation, mirror object semantic segmentation, and generic segmentation datasets. Our method outperforms the state-of-the-art by a large margin, achieving +4.2% mIoU on Trans10K-v2, +5.6% mIoU on MSD, +10.1% mIoU on RGBD-Mirror, +13.1% mIoU on TROSD, and +8.3% mIoU on Stanford2D3D, showing the effectiveness of our method against glass objects.

Problem

• Segmenting transparent objects using boundary and reflection cues
• Enhancing glass object segmentation via transformer-based architecture
• Improving semantic segmentation accuracy across diverse benchmark datasets

Innovation

• Pyramidal transformer encoder-decoder architecture for segmentation
• Boundary Feature Enhancement module to capture edge cues
• Reflection Feature Enhancement module to utilize reflective properties