Less but Better: Enabling Generalized Zero-shot Learning Towards Unseen Domains by Intrinsic Learning from Redundant LLM Semantics

2024-03-21

arXiv.org

AI Summary

This work addresses the inter-domain information asymmetry in Cross-Domain Generalized Zero-Shot Learning (CDGZSL), caused by redundant class semantics generated by Large Language Models (LLMs). We propose the first CDGZSL paradigm targeting joint recognition of both seen and unseen classes in previously unobserved domains. To this end, we introduce Meta Domain Alignment Semantic Refinement (MDASR): it leverages LLM-based semantic parsing to extract intrinsic cross-domain semantics, then jointly performs inter-class similarity alignment and meta-learning-driven unseen-class feature generation—thereby disentangling domain-specific semantics and preserving domain-invariant, shared semantics to construct a unified feature space. Our method achieves significant performance gains on Office-Home and Mini-DomainNet. Furthermore, we release the first LLM-generated cross-domain semantic benchmark, establishing a standardized evaluation framework for CDGZSL research.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Application Domains: Humanities & Computational Social Science
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Generalized zero-shot learning (GZSL) focuses on recognizing seen and unseen classes against domain shift problem (DSP) where data of unseen classes may be misclassified as seen classes. However, existing GZSL is still limited to seen domains. In the current work, we pioneer cross-domain GZSL (CDGZSL) which addresses GZSL towards unseen domains. Different from existing GZSL methods which alleviate DSP by generating features of unseen classes with semantics, CDGZSL needs to construct a common feature space across domains and acquire the corresponding intrinsic semantics shared among domains to transfer from seen to unseen domains. Considering the information asymmetry problem caused by redundant class semantics annotated with large language models (LLMs), we present Meta Domain Alignment Semantic Refinement (MDASR). Technically, MDASR consists of two parts: Inter-class Similarity Alignment (ISA), which eliminates the non-intrinsic semantics not shared across all domains under the guidance of inter-class feature relationships, and Unseen-class Meta Generation (UMG), which preserves intrinsic semantics to maintain connectivity between seen and unseen classes by simulating feature generation. MDASR effectively aligns the redundant semantic space with the common feature space, mitigating the information asymmetry in CDGZSL. The effectiveness of MDASR is demonstrated on the Office-Home and Mini-DomainNet, and we have shared the LLM-based semantics for these datasets as the benchmark.

Problem

• Addresses cross-domain generalized zero-shot learning (CDGZSL) for unseen domains.
• Mitigates information asymmetry caused by redundant LLM-annotated class semantics.
• Proposes Meta Domain Alignment Semantic Refinement (MDASR) for domain transfer.

Innovation

• Constructs common feature space across domains
• Eliminates non-intrinsic semantics via inter-class alignment
• Simulates feature generation for unseen-class connectivity