LLM-based Models for Detecting Emerging Topics in Service Feedback

2026-06-25

AI Summary

This study addresses the challenge of efficiently uncovering emerging service quality issues and potential inequities from massive multilingual user feedback in public services such as taxation. To this end, the authors propose a lightweight human–AI collaborative analysis framework that integrates fine-tuned and quantized large language models, text similarity computation, and statistical methods, augmented with an expert feedback loop to mitigate model hallucinations while enhancing computational efficiency. Evaluations by tax officials demonstrate that the approach significantly outperforms baseline models in multilingual topic identification, yielding results more aligned with expert judgment. The framework thus offers a scalable, equitable, and evidence-based foundation for public service decision-making.

Abstract

Enhancing the analysis of service feedback is essential for public sector organizations, particularly tax administrations, where trust and compliance depend on fair and effective service delivery. As feedback volumes grow, identifying emerging service quality issues and potential disparities across diverse populations becomes increasingly challenging. Traditional approaches often rely on manual review or static expert-defined indicators, limiting scalability and the ability to capture complex patterns in textual feedback. This paper presents a novel methodology that integrates large language models (LLMs), statistical techniques, and human-AI collaboration to improve multilingual customer feedback analysis. The primary objective is to detect emerging service quality topics that may also reveal potential inequities in service delivery. Our framework combines fine-tuned, quantized LLMs with expert oversight to produce accurate, computationally efficient, and context-aware analyses. The proposed approach was evaluated using similarity analysis and assessments from experienced tax officers, demonstrating stronger alignment with expert judgments than baseline models. By incorporating a human-in-the-loop framework, the methodology reduces LLM fabrication while improving the reliability and relevance of generated insights. The results demonstrate the practicality of combining LLMs with human expertise to support scalable, evidence-based decision-making in public sector organizations. This work contributes to the development of responsible AI systems that enhance service quality, responsiveness, fairness, and public trust through more effective analysis of multilingual customer feedback.

Problem

• emerging topics
• service feedback
• service quality
• inequities
• multilingual analysis

Innovation

• large language models
• human-in-the-loop
• emerging topic detection
• multilingual feedback analysis
• responsible AI