Enhanced Urdu Intent Detection with Large Language Models and Prototype-Informed Predictive Pipelines

2025-05-08

AI Summary

Urgent gaps exist in Urdu intent detection—despite its status as the world’s tenth most spoken language—particularly regarding generalization to unseen intent classes and scarcity of few-shot learning methods. To address this, we propose LLMPIA, the first end-to-end framework supporting both few-shot learning and cross-class intent recognition for Urdu. Our approach introduces a novel Urdu-specific contrastive learning enhancement paradigm and a prototype-guided attention mechanism, substantially improving generalization to unseen intent categories. Furthermore, we conduct the first systematic evaluation of six large language models and thirteen similarity computation methods on Urdu intent detection. On the ATIS and Web Queries benchmarks, LLMPIA achieves 83.28% and 76.23% F1-score under 4-way 1-shot settings, and 98.25% and 84.42% under 4-way 5-shot settings. In the in-distribution setting, it outperforms the state-of-the-art by +53.55% absolute F1.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Language and Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Multifarious intent detection predictors are developed for different languages, including English, Chinese and French, however, the field remains underdeveloped for Urdu, the 10th most spoken language. In the realm of well-known languages, intent detection predictors utilize the strategy of few-shot learning and prediction of unseen classes based on the model training on seen classes. However, Urdu language lacks few-shot strategy based intent detection predictors and traditional predictors are focused on prediction of the same classes which models have seen in the train set. To empower Urdu language specific intent detection, this introduces a unique contrastive learning approach that leverages unlabeled Urdu data to re-train pre-trained language models. This re-training empowers LLMs representation learning for the downstream intent detection task. Finally, it reaps the combined potential of pre-trained LLMs and the prototype-informed attention mechanism to create a comprehensive end-to-end LLMPIA intent detection pipeline. Under the paradigm of proposed predictive pipeline, it explores the potential of 6 distinct language models and 13 distinct similarity computation methods. The proposed framework is evaluated on 2 public benchmark datasets, namely ATIS encompassing 5836 samples and Web Queries having 8519 samples. Across ATIS dataset under 4-way 1 shot and 4-way 5 shot experimental settings LLMPIA achieved 83.28% and 98.25% F1-Score and on Web Queries dataset produced 76.23% and 84.42% F1-Score, respectively. In an additional case study on the Web Queries dataset under same classes train and test set settings, LLMPIA outperformed state-of-the-art predictor by 53.55% F1-Score.

Problem

• Developing Urdu intent detection using few-shot learning
• Leveraging unlabeled Urdu data for contrastive learning
• Combining LLMs and prototype attention for intent detection

Innovation

• Contrastive learning with unlabeled Urdu data
• Prototype-informed attention mechanism integration
• End-to-end LLMPIA pipeline for Urdu intent detection