KARLA: Knowledge-base Augmented Retrieval for Language Models

2026-06-25

AI Summary

This work addresses the limitations of large language models in dynamically updating factual knowledge, providing traceable outputs, and maintaining accuracy without massive parameter counts. The authors propose a retrieval-augmented generation approach that leverages special trigger tokens: during text generation, the model automatically inserts these tokens to query an external knowledge base in real time, thereby injecting up-to-date factual information. This method enables post-hoc correction of factual errors and supports result provenance without requiring model retraining. By integrating external knowledge on demand, the framework preserves the computational efficiency of smaller models while achieving the expressive capacity of larger ones. Experimental results demonstrate that the proposed approach significantly enhances both factual accuracy and interpretability in both short- and long-form text generation.

Abstract

We propose a new method that allows an LLM to automatically pull in factual knowledge from a knowledge base during token generation. This means that (1)~factual knowledge in the LLM output can be updated without retraining the LLM, (2)~facts in the LLM output can be traced to the knowledge base for transparency and explainability, and (3)~smaller models can achieve the same factual accuracy as larger models. Our core idea is to train the model to produce special tokens that trigger a query to the knowledge base. Our experiments show that our method improves factual grounding in both short and long-form generation, and allows factual revisions to take effect through KB edits rather than parameter updates.

Problem

• factual knowledge
• knowledge base
• language models
• factual grounding
• transparency

Innovation

• knowledge-base augmentation
• retrieval-augmented generation
• factual grounding
• trigger tokens
• model transparency