MultAttnAttrib: Training-Free Multimodal Attribution in Long Document Question Answering

2026-07-01

AI Summary

This work addresses the lack of efficient, training-free answer attribution methods in multimodal long-document question answering, a gap that undermines model trustworthiness and safety. The authors propose a novel training-free attribution approach that automatically locates source evidence for generated answers by analyzing attention weights from the prefilling phase of large language models, integrating carefully selected attention heads, adaptive threshold calibration, and multimodal alignment. To facilitate evaluation, they introduce MultAttrEval, the first fine-grained benchmark tailored for multimodal attribution in long documents. Experimental results demonstrate that the proposed method substantially outperforms existing techniques in both unimodal and multimodal settings, achieving attribution accuracy on par with state-of-the-art models such as GPT-5.4 while incurring only one-seventh the inference latency of prompt-engineering-based approaches.

Abstract

As grounded QA systems are increasingly deployed in AI assistants, accurately attributing generated answers to evidence is critical for user trust and model safety. While unimodal attributions have been explored in depth, the multimodal setting remains relatively under-researched. As a result, we introduce MultAttnAttrib, a training-free attribution-generation method that leverages a model's prefill pass, selected attention heads, and calibrated thresholds to locate source evidence within a document. To establish baseline results for the method, we introduce MultAttrEval, a complementary benchmark dataset annotated with fine-grained, ground-truth attributions for answer components grounded in multimodal source documents. To our knowledge, this is the first evaluation dataset designed specifically for multimodal attribution in long-form documents. Experimental results show that MultAttnAttrib consistently outperforms a variety of attribution-generation methods, including several strong prompting-based approaches and matches the latest frontier models such as GPT 5.4. Our method not only substantially improves attribution accuracy for both unimodal and multimodal attribution types, but also produces attributions at up to one-seventh of the direct inference latency compared to prompting on the same base model.

Problem

• multimodal attribution
• question answering
• evidence grounding
• long document
• answer attribution

Innovation

• training-free attribution
• multimodal question answering
• attention-based evidence localization
• long-document QA
• MultAttrEval benchmark