To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning

2026-04-23

AI Summary

This study addresses the poor generalization of Transformer models to unseen variable names in symbolic reasoning tasks, attributing this failure to representational collapse—where embeddings and unembedding vectors of unused tokens converge toward similar directions, thereby degrading representation quality. The work provides the first analysis of this phenomenon from the perspective of representational collapse, elucidating why heuristic strategies such as “active forgetting” are effective. It further proposes a systematic approach combining architectural modifications, data augmentation, and embedding freezing or periodic resetting. Experimental results demonstrate that this method substantially enhances the out-of-vocabulary generalization of decoder-only Transformers in propositional logic reasoning. Additionally, the study verifies on the Gemma 3 model that embeddings of unused tokens exhibit high mutual similarity, rendering them unsuitable as initializations for downstream fine-tuning.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Computer Vision: Visual Reasoning & Symbolic Representations
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We investigate the ability of decoder-only transformer models to perform abstract symbolic reasoning; specifically solving propositional logic reasoning problems given in-context. Previous work demonstrated that models fail to generalize to problems involving variable names that were not observed during training, and it was shown that one reason behind this is the difficulty of copying (or generating) unseen tokens. We show both theoretically and empirically that a particular representational collapse also has a crucial role: the unembeddings (last-layer weights) of unseen tokens collapse to nearly the same vector during training. The collapse makes distinguishing multiple unseen variables difficult for the model (especially when the embedding and unembedding parameters are shared), and provides a mechanistic explanation for the effectiveness of existing heuristic interventions like "active forgetting", which periodically reset the token (un)embeddings. Based on these observations, we devise a combination of techniques, involving a small architecture change facilitating copying, data diversity, and freezing or resetting (un)embeddings, that achieves generalization to unseen tokens. We support our claims with extensive controlled experiments on propositional logic reasoning problems. Beyond synthetic experiments, we also observe evidence of (un)embedding collapse in the open-weight models in the Gemma 3 family, which includes 99 unused tokens reserved for downstream use. Empirically we find that the correlated embeddings of these tokens are a poor initialization for finetuning applications.

Problem

• symbolic reasoning
• generalization
• unseen tokens
• transformer
• propositional logic

Innovation

• representation collapse
• unembedding collapse
• symbolic reasoning
• in-context learning
• token generalization