Flick: Few Labels Text Classification using K-Aware Intermediate Learning in Multi-Task Low-Resource Languages

2025-06-12

AI Summary

To address the few-shot performance bottleneck in low-resource language text classification—caused by high noise in pseudo-labels and difficulty in domain adaptation—this paper proposes a multi-task intermediate learning framework. Methodologically, it integrates multi-task pretraining, dynamic pseudo-label generation, intra-cluster consistency modeling, and K-aware intermediate representation distillation. Its key innovation is the first-ever single-cluster cohesion-driven adaptive K-aware pseudo-label refinement mechanism, which effectively suppresses noise propagation. Evaluated across 14 datasets spanning 14 low-resource languages—including Arabic, Urdu, and Setswana—the framework achieves an average accuracy improvement of 3.2–7.8% over state-of-the-art methods under few-shot settings. Results demonstrate substantial gains in model robustness and cross-domain generalization capability.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Training deep learning networks with minimal supervision has gained significant research attention due to its potential to reduce reliance on extensive labelled data. While self-training methods have proven effective in semi-supervised learning, they remain vulnerable to errors from noisy pseudo labels. Moreover, most recent approaches to the few-label classification problem are either designed for resource-rich languages such as English or involve complex cascading models that are prone to overfitting. To address the persistent challenge of few-label text classification in truly low-resource linguistic contexts, where existing methods often struggle with noisy pseudo-labels and domain adaptation, we propose Flick. Unlike prior methods that rely on generic multi-cluster pseudo-labelling or complex cascading architectures, Flick leverages the fundamental insight that distilling high-confidence pseudo-labels from a broader set of initial clusters can dramatically improve pseudo-label quality, particularly for linguistically diverse, low-resource settings. Flick introduces a novel pseudo-label refinement component, a departure from traditional pseudo-labelling strategies by identifying and leveraging top-performing pseudo-label clusters. This component specifically learns to distil highly reliable pseudo-labels from an initial broad set by focusing on single-cluster cohesion and leveraging an adaptive top-k selection mechanism. This targeted refinement process is crucial for mitigating the propagation of errors inherent in low-resource data, allowing for robust fine-tuning of pre-trained language models with only a handful of true labels. We demonstrate Flick's efficacy across 14 diverse datasets, encompassing challenging low-resource languages such as Arabic, Urdu, and Setswana, alongside English, showcasing its superior performance and adaptability.

Problem

• Improves few-label text classification in low-resource languages
• Reduces noisy pseudo-label errors in semi-supervised learning
• Enhances pseudo-label quality via adaptive cluster refinement

Innovation

• Uses K-aware intermediate learning for pseudo-label refinement
• Focuses on single-cluster cohesion for reliable pseudo-labels
• Employs adaptive top-k selection to mitigate error propagation