The Scaling Properties of Implicit Deductive Reasoning in Transformers

2026-05-05

Abstract

We investigate the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers. By systematically decorrelating provability from spurious features and enforcing algorithmic alignment, we find that in sufficiently deep models with a bidirectional prefix mask, implicit reasoning approaches explicit CoT performance across graph topologies and problem widths, though CoT remains necessary for depth extrapolation.

Problem

• implicit deductive reasoning
• Horn clauses
• Transformers
• scaling properties
• chain-of-thought

Innovation

• implicit deductive reasoning
• algorithmic alignment
• Horn clauses
• Chain-of-Thought (CoT)
• scaling properties