Incremental Summarization for Customer Support via Progressive Note-Taking and Agent Feedback

2025-10-08

AI Summary

To address low efficiency in customer service scenarios caused by frequent agent context switching and redundant dialogue review, this paper proposes a production-ready incremental dialogue summarization system. Methodologically, it integrates a fine-tuned Mixtral-8x7B large language model for continual generation of concise key points and a DeBERTa-based classifier for dynamic filtering of irrelevant content. It further introduces a novel agent-edit feedback loop: online capture of human revisions enables real-time summary refinement, while offline aggregation of edits supports iterative model improvement. The core contribution lies in modeling human editing signals as weak supervision to jointly align summary quality with user intent. Deployment results demonstrate an average 3% reduction in case handling time, up to 9% reduction for high-complexity cases, and a significant improvement in customer satisfaction.

Technology Category

• Natural Language Processing: Summarization
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Multiagent Systems: Agent Communication

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We introduce an incremental summarization system for customer support agents that intelligently determines when to generate concise bullet notes during conversations, reducing agents' context-switching effort and redundant review. Our approach combines a fine-tuned Mixtral-8x7B model for continuous note generation with a DeBERTa-based classifier to filter trivial content. Agent edits refine the online notes generation and regularly inform offline model retraining, closing the agent edits feedback loop. Deployed in production, our system achieved a 3% reduction in case handling time compared to bulk summarization (with reductions of up to 9% in highly complex cases), alongside high agent satisfaction ratings from surveys. These results demonstrate that incremental summarization with continuous feedback effectively enhances summary quality and agent productivity at scale.

Problem

• Determines when to generate concise notes during customer conversations
• Reduces agents' context-switching effort and redundant content review
• Enhances summary quality and agent productivity through continuous feedback

Innovation

• Progressive note-taking system reduces agent effort
• Fine-tuned Mixtral-8x7B model generates continuous notes
• DeBERTa-based classifier filters trivial conversation content