ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought

2026-01-30

AI Summary

This work addresses the performance degradation of existing implicit reasoning methods caused by the absence of effective compression guidance, while explicit chain-of-thought (CoT) approaches suffer from redundant computation. To overcome these limitations, the authors propose a novel implicit reasoning framework based on variational autoencoding, which— for the first time—leverages rendered CoT images as multimodal supervision signals. By extracting visual semantic representations from these images to regularize the posterior distribution, the method achieves highly efficient reasoning with minimal information loss. Extensive experiments demonstrate that the proposed approach significantly outperforms current implicit reasoning techniques across multiple tasks, offering not only superior computational efficiency but also reasoning performance that surpasses that of the original CoT.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

While Chain-of-Thought (CoT) significantly enhances the performance of Large Language Models (LLMs), explicit reasoning chains introduce substantial computational redundancy. Recent latent reasoning methods attempt to mitigate this by compressing reasoning processes into latent space, but often suffer from severe performance degradation due to the lack of appropriate compression guidance. In this study, we propose Rendered CoT-Guided variational Latent Reasoning (ReGuLaR), a simple yet novel latent learning paradigm resolving this issue. Fundamentally, we formulate latent reasoning within the Variational Auto-Encoding (VAE) framework, sampling the current latent reasoning state from the posterior distribution conditioned on previous ones. Specifically, when learning this variational latent reasoning model, we render explicit reasoning chains as images, from which we extract dense visual-semantic representations to regularize the posterior distribution, thereby achieving efficient compression with minimal information loss. Extensive experiments demonstrate that ReGuLaR significantly outperforms existing latent reasoning methods across both computational efficiency and reasoning effectiveness, and even surpasses CoT through multi-modal reasoning, providing a new and insightful solution to latent reasoning. Code: https://github.com/FanmengWang/ReGuLaR.

Problem

• latent reasoning
• Chain-of-Thought
• computational redundancy
• performance degradation
• reasoning compression

Innovation

• latent reasoning
• Chain-of-Thought
• Variational Auto-Encoding
• multi-modal regularization
• computational efficiency