Conformal Path Reasoning: Trustworthy Knowledge Graph Question Answering via Path-Level Calibration

2026-05-08

AI Summary

Existing knowledge graph question answering methods lack statistical coverage guarantees, and traditional conformal prediction approaches suffer from calibration failure and insufficient discriminability of path scores. To address these limitations, this work proposes a path-level conformal calibration mechanism coupled with a Residual Conformal Value Network (RCVNet) trained via PUCT-based exploration. The approach learns highly discriminative nonconformity scores while satisfying the exchangeability assumption, thereby enabling query-level conformal prediction. Empirical evaluation demonstrates that the method substantially improves both the compactness and coverage of predicted answer sets, achieving a 34% increase in empirical coverage and a 40% reduction in average prediction set size on benchmark datasets, significantly outperforming existing conformal baselines.

Abstract

Knowledge Graph Question Answering (KGQA) has shown promise for grounded and interpretable reasoning, yet existing approaches often fail to provide reliable coverage guarantees over retrieved answers. While Conformal Prediction (CP) offers a principled framework for producing prediction sets with statistical guarantees, prior methods suffer from critical limitations in both calibration validity and score discriminability, resulting in violated coverage guarantees and excessively large prediction sets. To address these pitfalls, we propose Conformal Path Reasoning (CPR), a trustworthy KGQA framework with two key innovations. First, we perform query-level conformal calibration over path-level scores, preserving the exchangeability while generating path prediction sets. Second, we introduce the Residual Conformal Value Network (RCVNet), a lightweight module trained via PUCT-guided exploration to learn discriminative path-level nonconformity scores. Experiments on benchmarks show that CPR significantly improves the Empirical Coverage Rate by 34% while reducing average prediction set size by 40% compared to conformal baselines. These results validate the efficacy of CPR in satisfying coverage guarantees with substantially more compact answer sets.

Problem

• Knowledge Graph Question Answering
• Conformal Prediction
• Coverage Guarantee
• Prediction Set
• Path Reasoning

Innovation

• Conformal Prediction
• Path-Level Calibration
• Knowledge Graph Question Answering
• Residual Conformal Value Network
• Coverage Guarantee