OLMoTrace: Tracing Language Model Outputs Back to Trillions of Training Tokens

2025-04-09

AI Summary

This paper addresses the challenge of tracing large language model (LLM) outputs back to their training data. We propose the first system enabling full-scale, real-time, fine-grained output provenance. Methodologically, it leverages an extended Infini-gram index integrated with efficient approximate string matching and a memory-optimized retrieval architecture, achieving millisecond-level, character-level alignment against trillion-token training corpora. Our contributions are threefold: (1) the first end-to-end, open-source, and reproducible system for real-time provenance across the entire training dataset; (2) substantially enhanced interpretability for factual verification, hallucination attribution, and creativity analysis; and (3) empirical validation on open models including OLMo, with query latency under several seconds. The system bridges a critical gap in LLM transparency, enabling precise, scalable, and operationally feasible attribution without compromising performance or accessibility.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We present OLMoTrace, the first system that traces the outputs of language models back to their full, multi-trillion-token training data in real time. OLMoTrace finds and shows verbatim matches between segments of language model output and documents in the training text corpora. Powered by an extended version of infini-gram (Liu et al., 2024), our system returns tracing results within a few seconds. OLMoTrace can help users understand the behavior of language models through the lens of their training data. We showcase how it can be used to explore fact checking, hallucination, and the creativity of language models. OLMoTrace is publicly available and fully open-source.

Problem

• Traces LM outputs to training data sources
• Identifies verbatim matches in training corpora
• Helps analyze factuality and model behavior

Innovation

• Real-time tracing of LM outputs to training data
• Uses extended infini-gram for fast results
• Open-source system for analyzing model behavior