SEER-ZSL: Semantic Encoder-Enhanced Representations for Generalized Zero-Shot Learning

2023-12-20

arXiv.org

AI Summary

To address the generalization bottleneck in generalized zero-shot learning (GZSL) caused by semantic-visual distribution misalignment, this paper proposes a collaborative framework integrating a probabilistic semantic encoder with adversarial visual distillation. The method enhances semantic consistency through probabilistic semantic modeling and jointly aligns visual manifolds via GAN-driven adversarial learning and semantic-visual co-distillation, thereby improving noise robustness and cross-domain generalization. Extensive experiments on small-, medium-, and large-scale benchmark datasets demonstrate that our approach consistently outperforms state-of-the-art methods, achieving significant gains in both seen and unseen class classification accuracy as well as generalization stability. Notably, it is the first work to jointly optimize semantic generation reliability and visual manifold fidelity, establishing a new paradigm for harmonizing semantic and visual representation learning in GZSL.

Abstract

Zero-Shot Learning (ZSL) presents the challenge of identifying categories not seen during training. This task is crucial in domains where it is costly, prohibited, or simply not feasible to collect training data. ZSL depends on a mapping between the visual space and available semantic information. Prior works learn a mapping between spaces that can be exploited during inference. We contend, however, that the disparity between meticulously curated semantic spaces and the inherently noisy nature of real-world data remains a substantial and unresolved challenge. In this paper, we address this by introducing a Semantic Encoder-Enhanced Representations for Zero-Shot Learning (SEER-ZSL). We propose a hybrid strategy to address the generalization gap. First, we aim to distill meaningful semantic information using a probabilistic encoder, enhancing the semantic consistency and robustness. Second, we distill the visual space by exploiting the learned data distribution through an adversarially trained generator. Finally, we align the distilled information, enabling a mapping of unseen categories onto the true data manifold. We demonstrate empirically that this approach yields a model that outperforms the state-of-the-art benchmarks in terms of both generalization and benchmarks across diverse settings with small, medium, and large datasets. The complete code is available on GitHub.

Problem

• Zero-Shot Learning
• Unseen Classes
• Semantic Information Gap

Innovation

• Probabilistic Encoder
• Adversarial Training
• Semantic Information Alignment