IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals

2026-09-01

AI Summary

为解决大型视觉语言模型生成内容的事实准确性问题，提出IntroConformal框架，利用模型自身内省信号提供无分布事实性保证。

Abstract

Large Vision-Language Models (LVLMs) have achieved strong multimodal performance, yet ensuring the factual correctness of generated content remains challenging. Existing methods that provide statistical guarantees on factuality typically rely on external verifiers or generation-time confidence signals, which introduce auxiliary dependencies or often fail for confident but incorrect outputs. We argue that reliable factuality control can instead be achieved through introspective signals derived from the model itself. We introduce IntroConformal, a training-free Conformal Risk Control (CRC) framework that provides finite-sample, distribution-free factuality guarantees. We first instantiate it with layer-wise semantic stability, a conformity score derived from hidden-state representations, and then propose verification probability, a stronger score capturing the model's self-administered judgment on claim factuality. Across multiple LVLM architectures, IntroConformal satisfies the conformal risk guarantee while substantially reducing abstention and achieving competitive or superior claim-level discrimination relative to external verifier-based baselines.

Problem

• Large Vision-Language Models
• Factuality
• Conformal Risk Control

Innovation

• IntroConformal
• conformal risk control
• introspective signals
• layer-wise semantic stability
• verification probability