AI Knowledge Assist: An Automated Approach for the Creation of Knowledge Bases for Conversational AI Agents

2025-10-09

AI Summary

To address the cold-start challenge in enterprise customer-service conversational AI—stemming from the absence of domain-specific knowledge bases—this paper proposes an end-to-end automated knowledge base construction framework. Leveraging historical customer-agent dialogues, it fine-tunes a lightweight LLaMA-3.1-8B model on private enterprise data to perform semantic-aware extraction and structured organization of high-quality question-answer pairs. Unlike conventional approaches reliant on large proprietary foundation models, our method operates efficiently with minimal computational overhead. Empirical evaluation across 20 enterprises demonstrates >90% accuracy in answering informational queries. The core contribution lies in the first integration of parameter-efficient fine-tuning with dialogue-level knowledge distillation, substantially enhancing the plug-and-play capability of retrieval-augmented generation (RAG) systems in knowledge-sparse scenarios—thereby bridging a critical technical gap in enterprise conversational AI cold-start deployment.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The utilization of conversational AI systems by leveraging Retrieval Augmented Generation (RAG) techniques to solve customer problems has been on the rise with the rapid progress of Large Language Models (LLMs). However, the absence of a company-specific dedicated knowledge base is a major barrier to the integration of conversational AI systems in contact centers. To this end, we introduce AI Knowledge Assist, a system that extracts knowledge in the form of question-answer (QA) pairs from historical customer-agent conversations to automatically build a knowledge base. Fine-tuning a lightweight LLM on internal data demonstrates state-of-the-art performance, outperforming larger closed-source LLMs. More specifically, empirical evaluation on 20 companies demonstrates that the proposed AI Knowledge Assist system that leverages the LLaMA-3.1-8B model eliminates the cold-start gap in contact centers by achieving above 90% accuracy in answering information-seeking questions. This enables immediate deployment of RAG-powered chatbots.

Problem

• Automatically builds knowledge bases from customer-agent conversations
• Eliminates cold-start problem in contact center AI deployment
• Generates accurate question-answer pairs using lightweight LLMs

Innovation

• Automated QA pair extraction from conversation logs
• Fine-tuning lightweight LLaMA model on internal data
• Achieves 90% accuracy eliminating cold-start gap