Scientific Logicality Enriched Methodology for LLM Reasoning: A Practice in Physics

2026-05-16

AI Summary

This work addresses the frequent neglect of logical rigor in scientific reasoning by current large language models, which undermines the reliability of their conclusions. The study introduces scientific logicality as a core evaluation dimension and systematically develops a logic-oriented assessment framework alongside a principled data sampling methodology. A high-logicality question dataset is constructed from physics literature to support this approach. Through a logic-guided training paradigm, the authors demonstrate significant improvements in both logical faithfulness and task performance across three mainstream large language models. These results establish that enhancing logical coherence plays a critical role in advancing the models’ capacity for solving scientific problems.

Abstract

With the continuous advancement of reasoning abilities in Large Language Models (LLMs), their application to scientific reasoning tasks has gained significant research attention. Current research primarily emphasizes boosting LLMs' performance on scientific QA benchmarks by training on larger, more comprehensive datasets with extended reasoning chains. However, these approaches neglect the essence of the scientific reasoning process -- logicality, which is the rational foundation to ensure the validity of reasoning steps leading to reliable conclusions. In this work, we make the first systematic investigation into the internal logicality underlying LLM scientific reasoning, and develop a scientific logicality-enriched methodology, including a set of assessment criteria and data sampling methods for logicality-guided training, to improve the logical faithfulness as well as task performance. Further, we take physics, characterized by its diverse logical structures and formalisms, as an exemplar discipline to practise the above methodology. For data construction, we extract scientific problems from academic literature and sample a high-quality dataset exhibiting strong logicality. Experiments based on three different backbone LLMs reveal that: 1) the training data we constructed can effectively improve the scientific logicality in LLM reasoning; and 2) the enriched scientific logicality plays a critical role in solving scientific problems. Code is available at \href{https://github.com/ScienceOne-AI/PhysLogic}{https://github.com/ScienceOne-AI/PhysLogic}.

Problem

• scientific reasoning
• logicality
• Large Language Models
• physics
• reasoning validity

Innovation

• scientific logicality
• logical faithfulness
• LLM reasoning
• physics reasoning
• logic-guided training