Label-semantics Aware Generative Approach for Domain-Agnostic Multilabel Classification

2025-06-07

AI Summary

To address weak domain generalization and high annotation costs in multi-label text classification, this paper proposes a domain-agnostic generative framework: predefined labels are mapped to natural language semantic descriptions, and the model directly generates these descriptions; a fine-tuned Sentence Transformer then aligns the generated outputs with ground-truth label descriptions. Our key contribution is the first formulation of labels as generative semantic units, coupled with a dual-objective joint optimization—cross-entropy loss for label identification and cosine similarity loss to enforce semantic fidelity of generated descriptions. The method is parameter-efficient and enables zero-shot cross-domain transfer without domain adaptation. Evaluated on multiple standard benchmarks, it achieves state-of-the-art performance, improving Micro-F1 and Macro-F1 by 13.94% and 24.85%, respectively, over the latest baseline.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The explosion of textual data has made manual document classification increasingly challenging. To address this, we introduce a robust, efficient domain-agnostic generative model framework for multi-label text classification. Instead of treating labels as mere atomic symbols, our approach utilizes predefined label descriptions and is trained to generate these descriptions based on the input text. During inference, the generated descriptions are matched to the pre-defined labels using a finetuned sentence transformer. We integrate this with a dual-objective loss function, combining cross-entropy loss and cosine similarity of the generated sentences with the predefined target descriptions, ensuring both semantic alignment and accuracy. Our proposed model LAGAMC stands out for its parameter efficiency and versatility across diverse datasets, making it well-suited for practical applications. We demonstrate the effectiveness of our proposed model by achieving new state-of-the-art performances across all evaluated datasets, surpassing several strong baselines. We achieve improvements of 13.94% in Micro-F1 and 24.85% in Macro-F1 compared to the closest baseline across all datasets.

Problem

• Develops domain-agnostic multi-label text classification model
• Uses label descriptions for semantic alignment and accuracy
• Achieves state-of-the-art performance with parameter efficiency

Innovation

• Generative model using label descriptions
• Sentence transformer for label matching
• Dual-objective loss for semantic alignment