LANID: LLM-assisted New Intent Discovery

2025-03-31

International Conference on Language Resources and Evaluation

AI Summary

To address insufficient semantic modeling, reliance on external knowledge, and high deployment costs of large language models (LLMs) in novel intent discovery (NID) for task-oriented dialogue systems, this paper proposes a lightweight, fully unsupervised LLM-driven NID framework. Our method leverages the zero-shot discriminative capability of LLMs to automatically extract high-confidence semantic triplets via KNN and DBSCAN, thereby constructing high-quality self-supervised signals; these triplets then guide the training of a lightweight text encoder under triplet loss. Evaluated on three standard NID benchmarks, our approach consistently outperforms strong baselines, achieving significant F1 improvements in both unsupervised and semi-supervised settings. The framework is computationally efficient, scalable, and practical for real-world deployment. Code is publicly available.

Technology Category

• Natural Language Processing: Information Extraction
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Data annotation is expensive in Task-Oriented Dialogue (TOD) systems. New Intent Discovery (NID) is a task aims to identify novel intents while retaining the ability to recognize known intents. It is essential for expanding the intent base of task-based dialogue systems. Previous works relying on external datasets are hardly extendable. Meanwhile, the effective ones are generally depends on the power of the Large Language Models (LLMs). To address the limitation of model extensibility and take advantages of LLMs for the NID task, we propose LANID, a framework that leverages LLM’s zero-shot capability to enhance the performance of a smaller text encoder on the NID task. LANID employs KNN and DBSCAN algorithms to select appropriate pairs of utterances from the training set. The LLM is then asked to determine the relationships between them. The collected data are then used to construct finetuning task and the small text encoder is optimized with a triplet loss. Our experimental results demonstrate the efficacy of the proposed method on three distinct NID datasets, surpassing all strong baselines in both unsupervised and semi-supervised settings. Our code can be found in https://github.com/floatSDSDS/LANID.

Problem

• Identifies new intents in dialogue systems
• Improves semantic representation using LLMs
• Enhances lightweight encoders via contrastive learning

Innovation

• LLM-guided lightweight encoder enhancement
• KNN and DBSCAN for selective utterance sampling
• Contrastive fine-tuning with triplet loss