Data-driven Machine Learning Cannot Reach Symbolic-level Logical Reasoning -- The Limit of the Scaling Law

2026-06-24

AI Summary

This study investigates whether data-driven machine learning can achieve symbolic-level syllogistic reasoning through scaling data and training time. Combining theoretical analysis with empirical validation, it systematically demonstrates for the first time that supervised learning fundamentally fails to realize rigorous symbolic reasoning due to insufficient coverage of training data and an inherent contradiction in end-to-end mapping. Experiments reveal that Euler Net cannot distinguish all 24 valid syllogistic forms, while GPT-5–series models—despite achieving 100% accuracy across various surface representations (e.g., natural language, bigrams, symbols)—frequently produce incorrect explanations, indicating that high accuracy does not equate to reliable reasoning. The findings expose fundamental limitations of deep learning in symbolic reasoning and delineate the boundaries of scaling laws.

Abstract

Sphere neural networks have achieved symbolic level syllogistic reasoning without training data, raising the question of where the limit of the scaling law for logical reasoning lies, i.e., whether data-driven machine learning systems can achieve the same level by increasing training data and training time. We show two methodological limitations that prevent supervised deep learning from reaching the symbolic-level syllogistic reasoning: (1) training data can not distinguish all 24 types of valid syllogistic reasoning; (2) end-to-end mapping from premises to conclusion introduces contradictory training targets between neural components for pattern recognition and logical reasoning. Beside theoretical analysis, we experimentally illustrate that Euler Net cannot achieve rigorous syllogistic reasoning. We further challenge the most recent ChatGPTs (GPT-5-nano and GPT-5) to determine the satisfiability of syllogistic statements in four surface forms (patterns): words, double words, simple symbols, and long random symbols, showing that surface forms affect the reasoning performance and that ChatGPT GPT-5 may reach 100% accuracy but still provide incorrect explanations. As empirical training processes are stopped after achieving 100% accuracy, we conclude that supervised machine learning systems will not attain the rigour of symbolic logical reasoning.

Problem

• symbolic reasoning
• scaling law
• syllogistic reasoning
• machine learning
• logical reasoning

Innovation

• symbolic reasoning
• syllogistic reasoning
• scaling law
• neural-symbolic gap
• logical consistency