Conversational Retrieval and On-the-Fly Knowledge Modeling of Historical Penitentiary Repression Records

2026-07-09

AI Summary

Existing retrieval-augmented generation (RAG) systems struggle to provide holistic interpretations of historical digital collections and lack the ability to dynamically integrate expert knowledge for complex queries. This work proposes a conversational document analysis system that, for the first time, combines dynamic knowledge graphs with conversational RAG. During user interactions, the system incrementally constructs a graph structure that fuses archival expert knowledge with retrieved results, serving as external memory for the language model. By moving beyond the traditional RAG paradigm—which relies solely on raw documents—this approach effectively models cross-document relationships, long-range dependencies, and implicit knowledge, substantially enhancing the system’s capacity to answer multi-record complex questions and deliver deeper historical interpretations.

Abstract

Recent developments in digital libraries increasingly favor conversational and natural language access to information through Retrieval-Augmented Generation (RAG). Although these approaches are effective for extractive tasks grounded in individual records, they remain limited in their ability to interpret document collections holistically and to incorporate expert knowledge dynamically. In this article, we present a document analysis system designed for the management of historical digital libraries that supports on-the-fly knowledge modeling. The system is equipped with the capability to store facts produced either by expert archivists or derived from document retrieval processes within a graph-based structure. Through continuous professional interaction, the system can retrieve information not only from primary sources such as documents, but also from previously modeled knowledge, with the graph-based index acting as a memory for the language model to access. This enables increasingly complex queries involving long-term dependencies across documents, link discovery, and the integration of expert knowledge that may not be explicitly present in the original sources. As a result, the proposed approach facilitates the generation of richer and more comprehensive information.

Problem

• Conversational Retrieval
• Knowledge Modeling
• Historical Digital Libraries
• Expert Knowledge Integration
• Document Collection Interpretation

Innovation

• Retrieval-Augmented Generation
• knowledge graph
• on-the-fly knowledge modeling
• conversational retrieval
• historical digital libraries