XPrompt:Explaining Large Language Model's Generation via Joint Prompt Attribution

2024-05-30

arXiv.org

AI Summary

This study addresses the multi-prompt collaborative attribution problem in large language model (LLM) text generation: how multiple input prompts jointly influence the full output, rather than being attributed in isolation to individual tokens or spans. To this end, we propose the first joint prompt attribution framework, formulating attribution as a causal combinatorial optimization problem over a discrete space. We design a counterfactual reasoning–based probabilistic search algorithm that balances explanation fidelity and computational efficiency. We establish a multidimensional evaluation framework assessing fidelity, stability, and efficiency, and validate our method across diverse LLMs and generation tasks. Compared to baseline approaches, our method achieves an average 23.6% improvement in attribution fidelity and accelerates inference by 5.8×. This work introduces a scalable, verifiable paradigm for LLM input sensitivity analysis and controllable generation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Large Language Models (LLMs) have demonstrated impressive performances in complex text generation tasks. However, the contribution of the input prompt to the generated content still remains obscure to humans, underscoring the necessity of elucidating and explaining the causality between input and output pairs. Existing works for providing prompt-specific explanation often confine model output to be classification or next-word prediction. Few initial attempts aiming to explain the entire language generation often treat input prompt texts independently, ignoring their combinatorial effects on the follow-up generation. In this study, we introduce a counterfactual explanation framework based on joint prompt attribution, XPrompt, which aims to explain how a few prompt texts collaboratively influences the LLM's complete generation. Particularly, we formulate the task of prompt attribution for generation interpretation as a combinatorial optimization problem, and introduce a probabilistic algorithm to search for the casual input combination in the discrete space. We define and utilize multiple metrics to evaluate the produced explanations, demonstrating both faithfulness and efficiency of our framework.

Problem

• Explains collaborative influence of prompts on LLM generation
• Addresses combinatorial optimization for causal input attribution
• Interprets complete text generation beyond classification tasks

Innovation

• Joint Prompt Attribution framework
• Counterfactual explanation for generation
• Combinatorial optimization probabilistic algorithm