SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception

2026-07-12

AI Summary

This work addresses the inconsistency in spatial attention caused by lexical variation among synonymous expressions in open-vocabulary dense perception. To resolve this issue, the authors propose a synonym-consistent language–image pretraining framework built upon the CLIP architecture, incorporating two key components: Semantic-consistent Spatial Attention alignment (SSA) and Spatial Attention Refinement (SAR). They also construct SEViC, the first visual corpus annotated with multiple synonyms and their definitions, enabling the model to learn consistent spatial attention for semantically equivalent textual descriptions. Experimental results demonstrate that the proposed method significantly enhances localization robustness on unseen categories and achieves state-of-the-art performance among CLIP-based approaches across multiple open-vocabulary dense perception benchmarks.

Abstract

Open-vocabulary dense perception (OVDP) aims to localize objects unseen during training by leveraging textual knowledge. Despite the remarkable progress of recent CLIP-based approaches, we identify a critical limitation: synonym-induced grounding inconsistency, where semantically equivalent expressions yield disparate spatial attention patterns. This inconsistency undermines the robustness and performance of existing methods in real-world OVDP applications. To address this issue, we propose SynCLIP, a Synonym-Coherent Language-Image Pretraining framework that enhances synonym-robust grounding for OVDP. SynCLIP introduces a Semantic-consistent Spatial Attention alignment (SSA) module to enhance spatial attention consistency by minimizing discrepancies between attention maps of original and synonymous expressions. Furthermore, a Spatial Attention Refinement (SAR) module selectively strengthens the most semantically relevant spatial regions within aligned maps for more precise and stable grounding. To support synonym-coherent pretraining, we also construct a Synonym-Enriched Visual Corpus (SEViC), which augments each category with multiple synonyms and textual definitions. Extensive experiments on multiple benchmarks demonstrate that SynCLIP substantially improves grounding consistency under diverse linguistic variants and achieves state-of-the-art performance among CLIP-based OVDP methods. Code is available at https://github.com/Justlovesmile/SynCLIP.

Problem

• open-vocabulary dense perception
• synonym-induced grounding inconsistency
• spatial attention consistency
• language-image pretraining
• semantic equivalence

Innovation

• Synonym-Coherent Pretraining
• Spatial Attention Alignment
• Open-Vocabulary Dense Perception
• Semantic Consistency
• CLIP-based Grounding