Cross-Examination Framework: A Task-Agnostic Diagnostic for Information Fidelity in Text-to-Text Generation
Existing text generation evaluation metrics, such as BLEU and BERTScore, struggle to effectively assess semantic fidelity and often overlook critical errors like content omissions or factual inconsistencies. This work proposes a reference-free, multidimensional evaluation framework that introduces a cross-examination mechanism into generation assessment for the first time: treating the source and generated texts as independent knowledge bases, it performs question-answer–based mutual validation by generating verifiable questions from each to interrogate the other. The method yields three interpretable scores—coverage, consistency, and conformity—without requiring reference texts, enabling precise detection of semantic distortions. It significantly outperforms conventional metrics across translation, summarization, and clinical note tasks, effectively capturing errors at the entity and relational levels. Its reference-free and reference-based modes exhibit strong correlation, and expert validation confirms that mismatched questions align closely with actual semantic errors.