Do LLMs Game Formalization? Evaluating Faithfulness in Logical Reasoning

2026-04-21

AI Summary

This study addresses the "formalization gaming" problem, wherein large language models generate syntactically valid but semantically distorted axiomatizations when translating natural language logic problems into Lean 4 proofs. The work systematically distinguishes two types of unfaithful behavior: mistranslation of premises during formalization and fabrication of axioms during proof generation. To evaluate semantic fidelity, the authors propose a decoupled two-stage pipeline—separating formalization from proof—and integrate cross-stage consistency checks with adversarial prompting. Evaluations on 303 first-order logic problems reveal that high compilation rates do not guarantee semantic faithfulness; while end-to-end generation shows no systematic gaming, the two-stage approach uncovers both detectable and latent unfaithful patterns in GPT-5 and DeepSeek-R1. These findings challenge the validity of relying solely on compilation or accuracy metrics to assess logical reasoning capabilities.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: Generation
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Formal verification guarantees proof validity but not formalization faithfulness. For natural-language logical reasoning, where models construct axiom systems from scratch without library constraints, this gap between valid proofs and faithful translations is especially acute. We investigate whether frontier models exploit this gap when generating Lean 4 proofs, a behavior we term formalization gaming. We evaluate GPT-5 and DeepSeek-R1 on 303 first-order logic problems (203 from FOLIO, 100 from Multi-LogiEval), comparing unified generation against a two-stage pipeline that separates formalization from proving. Despite compilation rates of 87-99%, we find no evidence of systematic gaming in unified generation: models prefer reporting failure over forcing proofs, even under prompting designed to encourage it. However, unfaithfulness that evades our detection signals may still occur. The two-stage pipeline reveals two distinct modes of unfaithfulness: GPT-5 fabricates axioms during proof generation, a reactive fallback detectable via cross-stage comparison, while DeepSeek-R1 mistranslates premises during formalization, producing internally consistent outputs that evade detection entirely. These findings show that high compilation rates or accuracies should not be equated with faithful reasoning. Code and data are available at https://github.com/koreankiwi99/formalization-gaming.

Problem

• formalization faithfulness
• logical reasoning
• large language models
• formal verification
• natural language to formal logic

Innovation

• formalization gaming
• faithful reasoning
• two-stage pipeline
• logical formalization
• LLM evaluation