TracLLM: A Generic Framework for Attributing Long Context LLMs

2025-06-04

AI Summary

Contextual backtracking aims to identify the critical input segments (e.g., sentences or paragraphs) upon which long-context large language model (LLM) outputs depend—crucial for retrieval-augmented generation (RAG), agent interpretability, forensic analysis of prompt injection/knowledge contamination attacks, and user trust building. This paper introduces the first general-purpose attribution framework for long-context LLMs. It integrates heuristic search for computational efficiency with Shapley value optimization, multi-source contribution ensemble, and adaptive denoising to improve attribution accuracy. Evaluated across diverse long-context tasks, our method significantly enhances both precision in identifying supporting evidence and inference efficiency. It robustly supports system debugging, adversarial attribution, and trustworthy reasoning. By unifying efficiency and fidelity in input attribution, the framework establishes a novel paradigm for explainability in long-context LLMs.

Technology Category

• Search and Optimization: Learning to Search
• Multiagent Systems: Adversarial Agents
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Long context large language models (LLMs) are deployed in many real-world applications such as RAG, agent, and broad LLM-integrated applications. Given an instruction and a long context (e.g., documents, PDF files, webpages), a long context LLM can generate an output grounded in the provided context, aiming to provide more accurate, up-to-date, and verifiable outputs while reducing hallucinations and unsupported claims. This raises a research question: how to pinpoint the texts (e.g., sentences, passages, or paragraphs) in the context that contribute most to or are responsible for the generated output by an LLM? This process, which we call context traceback, has various real-world applications, such as 1) debugging LLM-based systems, 2) conducting post-attack forensic analysis for attacks (e.g., prompt injection attack, knowledge corruption attacks) to an LLM, and 3) highlighting knowledge sources to enhance the trust of users towards outputs generated by LLMs. When applied to context traceback for long context LLMs, existing feature attribution methods such as Shapley have sub-optimal performance and/or incur a large computational cost. In this work, we develop TracLLM, the first generic context traceback framework tailored to long context LLMs. Our framework can improve the effectiveness and efficiency of existing feature attribution methods. To improve the efficiency, we develop an informed search based algorithm in TracLLM. We also develop contribution score ensemble/denoising techniques to improve the accuracy of TracLLM. Our evaluation results show TracLLM can effectively identify texts in a long context that lead to the output of an LLM. Our code and data are at: https://github.com/Wang-Yanting/TracLLM.

Problem

• Pinpointing texts in long contexts that influence LLM outputs
• Improving efficiency and accuracy of context traceback methods
• Enhancing trust and debugging in LLM-based applications

Innovation

• Generic context traceback framework for LLMs
• Informed search algorithm enhances efficiency
• Contribution score ensemble improves accuracy