Building a Few-Shot Cross-Domain Multilingual NLU Model for Customer Care

2025-06-04

European Conference on Artificial Intelligence

AI Summary

To address the limited labeled data in cross-channel, cross-regional, and multilingual e-commerce customer service scenarios—which severely constrains intent classification performance—this paper proposes a few-shot cross-domain multilingual intent recognition framework. Methodologically, we design an integrated encoder-classifier architecture that jointly applies isotropic regularization fine-tuning and multilingual knowledge distillation to enhance cross-domain generalization and cross-lingual semantic consistency in the embedding space. Experiments on a bilingual (English–Spanish) customer service dataset from Canada and Mexico demonstrate that our approach achieves 20–23% higher accuracy than state-of-the-art pretrained models using only 3–5 labeled examples per intent class, substantially reducing adaptation cost for new domains. To the best of our knowledge, this is the first work to synergistically incorporate isotropic constraints and multilingual distillation into few-shot cross-domain NLU. Our framework establishes a scalable, highly generalizable modeling paradigm for low-resource multilingual customer service systems.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Customer care is an essential pillar of the e-commerce shopping experience with companies spending millions of dollars each year, employing automation and human agents, across geographies (like US, Canada, Mexico, Chile), channels (like Chat, Interactive Voice Response (IVR)), and languages (like English, Spanish). SOTA pre-trained models like multilingual-BERT, fine-tuned on annotated data have shown good performance in downstream tasks relevant to Customer Care. However, model performance is largely subject to the availability of sufficient annotated domain-specific data. Cross-domain availability of data remains a bottleneck, thus building an intent classifier that generalizes across domains (defined by channel, geography, and language) with only a few annotations, is of great practical value. In this paper, we propose an embedder-cum-classifier model architecture which extends state-of-the-art domain-specific models to other domains with only a few labeled samples. We adopt a supervised fine-tuning approach with isotropic regularizers to train a domain-specific sentence embedder and a multilingual knowledge distillation strategy to generalize this embedder across multiple domains. The trained embedder, further augmented with a simple linear classifier can be deployed for new domains. Experiments on Canada and Mexico e-commerce Customer Care dataset with few-shot intent detection show an increase in accuracy by 20-23% against the existing state-of-the-art pre-trained models.

Problem

• Develop few-shot cross-domain multilingual NLU model for customer care
• Overcome data scarcity across domains (channel, geography, language)
• Improve intent detection accuracy with minimal annotated data

Innovation

• Embedder-classifier model for cross-domain generalization
• Isotropic regularizers in supervised fine-tuning
• Multilingual knowledge distillation for domain adaptation