Building Specialized Software-Assistant ChatBot with Graph-Based Retrieval-Augmented Generation

2025-11-07

AI Summary

To address the dual challenges of high manual effort in constructing interactive software guides for enterprises and the hallucination-prone, hard-to-finetune nature of large language models (LLMs), this paper proposes a retrieval-augmented generation (RAG) framework grounded in a state-action knowledge graph. Our method automatically parses web interfaces to model complex enterprise systems—such as CRM and ERP platforms—as structured, queryable knowledge graphs, enabling context-aware navigation and precise reasoning over black-box LLMs without fine-tuning. Key contributions include: (1) the first formulation of dynamic UI states and user actions as a jointly modeled, retrievable graph structure; and (2) the integration of graph-based retrieval with RAG to enhance both interpretability and robustness of generated guidance. The framework has been deployed in production learning platforms RAKAM and Lemon Learning. Empirical evaluation demonstrates its high scalability and operational effectiveness in real-world industrial settings.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Digital Adoption Platforms (DAPs) have become essential tools for helping employees navigate complex enterprise software such as CRM, ERP, or HRMS systems. Companies like LemonLearning have shown how digital guidance can reduce training costs and accelerate onboarding. However, building and maintaining these interactive guides still requires extensive manual effort. Leveraging Large Language Models as virtual assistants is an appealing alternative, yet without a structured understanding of the target software, LLMs often hallucinate and produce unreliable answers. Moreover, most production-grade LLMs are black-box APIs, making fine-tuning impractical due to the lack of access to model weights. In this work, we introduce a Graph-based Retrieval-Augmented Generation framework that automatically converts enterprise web applications into state-action knowledge graphs, enabling LLMs to generate grounded and context-aware assistance. The framework was co-developed with the AI enterprise RAKAM, in collaboration with Lemon Learning. We detail the engineering pipeline that extracts and structures software interfaces, the design of the graph-based retrieval process, and the integration of our approach into production DAP workflows. Finally, we discuss scalability, robustness, and deployment lessons learned from industrial use cases.

Problem

• Automating interactive guide creation for enterprise software navigation
• Preventing LLM hallucinations in software assistance without fine-tuning
• Converting web applications into structured knowledge for reliable guidance

Innovation

• Graph-based Retrieval-Augmented Generation framework for enterprise software
• Automatically converts web applications into state-action knowledge graphs
• Enables LLMs to generate grounded context-aware assistance