Optimizing Decomposition for Optimal Claim Verification

2025-03-19

AI Summary

Existing long-text fact verification methods decouple claim decomposition from factual validation, leading to misalignment between decomposition granularity and the verifier’s preference for information density (atomicity). Method: This paper formally defines atomicity to quantify decomposition atomicity and proposes a dynamic decomposition framework that adaptively adjusts granularity based on verifier feedback. It models decomposition strategy optimization as a bilevel optimization problem and devises a reinforcement learning–based solution paradigm. Contribution/Results: The method enables joint optimization of decomposition and verification. Empirical evaluation across multiple verifiers, diverse datasets, and varying atomicity levels shows average improvements of +0.07 in verification confidence and +0.12 in accuracy (on a 0–1 scale), significantly mitigating the pipeline disconnection issue inherent in prior decoupled approaches.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Current research on the extit{Decompose-Then-Verify} paradigm for evaluating the factuality of long-form text typically treats decomposition and verification in isolation, overlooking their interactions and potential misalignment. We find that existing decomposition policies, typically hand-crafted demonstrations, do not align well with downstream verifiers in terms of atomicity -- a novel metric quantifying information density -- leading to suboptimal verification results. We formulate finding the optimal decomposition policy for optimal verification as a bilevel optimization problem. To approximate a solution for this strongly NP-hard problem, we propose dynamic decomposition, a reinforcement learning framework that leverages verifier feedback to learn a policy for dynamically decomposing claims to verifier-preferred atomicity. Experimental results show that dynamic decomposition outperforms existing decomposition policies, improving verification confidence by 0.07 and accuracy by 0.12 (on a 0-1 scale) on average across varying verifiers, datasets, and atomcities of input claims.

Problem

• Optimizing decomposition policy for better claim verification
• Aligning decomposition atomicity with verifier preferences
• Solving bilevel optimization for dynamic decomposition framework

Innovation

• Reinforcement learning framework for dynamic decomposition
• Bilevel optimization to align decomposition with verification
• Verifier feedback improves atomicity and accuracy