Problem
Research questions and friction points this paper is trying to address.
multi-label classification
incomplete annotations
prior assumptions
semantic mistakes
catastrophic overfitting
Adapting Dense Vision-Language Relationships for Multi-label Classification with Partial Label
🤖 AI Summary
本文针对多标签分类中的不完全标注问题，提出了一种语言驱动的密集语义适配器(LDSA)，利用预训练CLIP模型挖掘适应性关系，以减少过拟合并提高分类准确性。
📝 Abstract
Learning multi-label image classification with incomplete annotations is a challenging task that has been widely studied for its superior trade-off between high efficiency and less labor consumption on large-scale datasets. Predominant methods rely on strong prior assumptions to recover the missing semantics from partial annotations. However, these statistic priors suffer from unstable semantic mistakes and thus lead to catastrophic overfitting. Toward this end, we propose a Language-driven Dense Semantic Adaptor (LDSA) that excavates prior-adaptive relationships from multimodal pretrained CLIP models. In our approach, the densely contrastive adaptor is first proposed to construct dense visual contrastive constraints, transferring the task-specific knowledge to visual domains. We then propose a language-driven interactive decoder with the help of class-specific prompt tuning, which adapts language proxies with visual domains. With the collaborative learning of proposed modules, experimental results demonstrate our proposed LDSA achieves a new state of the art on public multi-label classification benchmarks, and interpretable analyses reveal that our LDSA discovers implicit semantic relationships with the prior-adaptive learning scheme.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Language-driven Dense Semantic Adaptor (LDSA)
partial label learning
multi-label image classification
dense visual contrastive constraints
class-specific prompt tuning
🔎 Similar Papers
No similar papers found.
