SafePassage: High-Fidelity Information Extraction with Black Box LLMs
Black-box large language models (LLMs) offer ease of deployment for information extraction (IE) but suffer from low reliability due to hallucination—generating outputs inconsistent with the source document. To address this, we propose SafePassage, a framework that enhances the fidelity of black-box LLM extractions by generating verifiable, document-grounded “safe passages.” Our method comprises three stages: (1) initial LLM-based extraction, (2) string-based global alignment and re-ranking to enforce strict textual consistency, and (3) lightweight fine-tuning of a compact encoder for scoring and final ranking. Crucially, SafePassage replaces end-to-end hallucination suppression with controllable generation and reveals that small fine-tuned encoders substantially outperform large models on passage-scoring tasks. Experiments show an 85% reduction in hallucination rate, high agreement with human judgments (Cohen’s κ = 0.91), and full training feasibility with only 1–2 hours of human annotation.