Reasoning Primitives in Hybrid and Non-Hybrid LLMs

2026-04-23

AI Summary

This study investigates whether the reasoning capabilities of large language models stem from fundamental operations such as memory retrieval and state tracking, and evaluates whether hybrid architectures—combining attention mechanisms with recurrent state updates—outperform pure Transformer models on tasks requiring both capacities. Through carefully controlled experiments, the authors systematically compare instruction-tuned and reasoning-augmented variants of Olmo3, implemented in both pure Transformer and hybrid forms. Results demonstrate that reasoning augmentation substantially extends the effective working range of models, while hybrid architectures exhibit greater robustness on tasks with strong sequential dependencies and high complexity, in contrast to pure Transformers whose performance degrades sharply as task difficulty increases. The work proposes decomposing reasoning into basic computational primitives, offering a novel perspective for understanding and enhancing model reasoning abilities.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Reasoning in large language models is often treated as a monolithic capability, but its observed gains may arise from more basic operations. We study reasoning through two such primitives, recall and state-tracking, and ask whether hybrid architectures that combine attention-based retrieval with recurrent state updates are better suited than attention-only models for tasks that jointly require both. Using matched Olmo3 transformer and hybrid models in instruction-tuned and reasoning-augmented variants, we evaluate these models on a set of controlled tasks involving a mixture of state-tracking and recall primitives, state-based recall. Across tasks, we notice that reasoning augmentation provides the largest overall improvement, substantially extending the range of difficulty over which models remain effective. We also notice that in certain tasks, the hybrid reasoning model remains substantially more robust as sequential dependence increases. In contrast, the transformer reasoning model degrades sharply in performance as task difficulty increases beyond a given threshold. These results suggest that reasoning tokens and architectural inductive biases contribute at different levels of the computational process: explicit reasoning can expand a model's effective operating range, but its benefit depends on how well the underlying architecture supports persistent state propagation. Given the small size of our case study, which involves a limited set of models and tasks, we present these findings as suggestive rather than conclusive and leave broader validation across model families, scales, and task variations to future work.

Problem

• reasoning primitives
• recall
• state-tracking
• hybrid architectures
• large language models

Innovation

• reasoning primitives
• hybrid architecture
• state-tracking
• recall
• reasoning augmentation