Context-Informed Grounding Supervision

2025-06-18

AI Summary

To address the tendency of large language models (LLMs) to over-rely on internal parametric knowledge—leading to factual hallucinations and poor grounding—in external knowledge-augmented settings, this paper proposes a context-aware grounding supervision training paradigm. Methodologically, it introduces a “context-preceding” token-level loss masking mechanism that computes supervision signals exclusively for tokens in the model’s response that are explicitly supported by external contextual evidence, thereby implicitly steering the model to prioritize input-grounded generation. The approach requires no architectural modifications and seamlessly integrates into standard post-training pipelines. Furthermore, we establish a cross-modal (text + vision) grounding evaluation framework. Experiments demonstrate significant improvements in textual grounding across 11 information retrieval benchmarks; in vision-language tasks, hallucination rates decrease and factual consistency improves, while general-purpose capabilities remain intact.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Large language models (LLMs) are often supplemented with external knowledge to provide information not encoded in their parameters or to reduce hallucination. In such cases, we expect the model to generate responses by grounding its response in the provided external context. However, prior work has shown that simply appending context at inference time does not ensure grounded generation. To address this, we propose Context-INformed Grounding Supervision (CINGS), a post-training supervision in which the model is trained with relevant context prepended to the response, while computing the loss only over the response tokens and masking out the context. Our experiments demonstrate that models trained with CINGS exhibit stronger grounding in both textual and visual domains compared to standard instruction-tuned models. In the text domain, CINGS outperforms other training methods across 11 information-seeking datasets and is complementary to inference-time grounding techniques. In the vision-language domain, replacing a vision-language model's LLM backbone with a CINGS-trained model reduces hallucinations across four benchmarks and maintains factual consistency throughout the generated response. This improved grounding comes without degradation in general downstream performance. Finally, we analyze the mechanism underlying the enhanced grounding in CINGS and find that it induces a shift in the model's prior knowledge and behavior, implicitly encouraging greater reliance on the external context.

Problem

• Ensures LLMs ground responses in external context
• Reduces hallucinations in vision-language model outputs
• Improves grounding without degrading general performance

Innovation

• Uses context prepended to response training
• Computes loss only over response tokens
• Encourages reliance on external context