Large Language Models' Reasoning Stalls: An Investigation into the Capabilities of Frontier Models

2025-05-26

AI Summary

This study investigates whether state-of-the-art large language models (LLMs) have made fundamental advances in automated theorem proving (ATP) reasoning strategies between December 2023 and August 2024, using the PRONTOQA steamroller benchmark. Method: We conduct a systematic evaluation via token-level completion tracking, response accuracy assessment, and conclusion relevance analysis, comparing forward chaining, backward chaining, and other ATP strategies. Contribution/Results: We provide the first empirical evidence that LLMs’ logical reasoning capabilities did not undergo qualitative improvement over this nine-month period; observed performance gains stem primarily from implicit system prompts or generic chain-of-thought training. Forward chaining emerges as the most effective ATP strategy for current LLMs. Crucially, we find only a weak positive correlation between correct reasoning traces and correct conclusions—revealing a critical “reasoning-result decoupling” bottleneck. These findings establish a new evaluation benchmark and yield theoretical insights for LLM reasoning assessment and ATP-LLM co-design.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Empirical methods to examine the capability of Large Language Models (LLMs) to use Automated Theorem Prover (ATP) reasoning strategies are studied. We evaluate the performance of State of the Art models from December 2023 and August 2024 on PRONTOQA steamroller reasoning problems. For that, we develop methods for assessing LLM response accuracy and correct answer correlation. Our results show that progress in improving LLM reasoning abilities has stalled over the nine month period. By tracking completion tokens, we show that almost all improvement in reasoning ability since GPT-4 was released can be attributed to either hidden system prompts or the training of models to automatically use generic Chain of Thought prompting strategies. Among the ATP reasoning strategies tried, we found that current frontier LLMs are best able to follow the bottom-up (also known as forward-chaining) strategy. A low positive correlation was found between an LLM response containing correct reasoning and arriving at the correct conclusion.

Problem

• Investigating stalled progress in LLM reasoning capabilities
• Evaluating LLM performance on ATP-based reasoning problems
• Assessing correlation between correct reasoning and conclusions

Innovation

• Evaluated LLMs using Automated Theorem Prover strategies
• Assessed response accuracy and correct answer correlation
• Found best performance with bottom-up reasoning strategy