Scaffolding Minds: Optimizing Latent Visual Target Representations for Multimodal Reasoning

2026-08-20

AI Summary

本文针对多模态推理中潜变量表示的优化问题，提出Scaffolding Minds方法，通过学习专用编码器和RL采样器的均值与方差来解决现有框架在监督微调及强化学习阶段存在的局限。

Abstract

Latent reasoning has advanced multimodal reasoning through a two-stage training paradigm: (1) a helper image is encoded into latent tokens to teach visual chain-of-thought during a supervised fine-tuning (SFT) stage, and (2) these latent tokens are further refined with reward feedback during a reinforcement learning (RL) stage. In this paper, we identify two key limitations of this framework, one in each stage. First, the SFT stage typically relies on an off-the-shelf vision encoder to encode the helper image, yielding suboptimal latent representations that may not be well aligned with the downstream reasoning task. Second, existing RL methods treat the latent component only through deterministic regularization, which constrains policy drift but does not create alternative latent trajectories for exploration. To address these limitations, we propose Scaffolding Minds. Our approach learns a dedicated scaffolding encoder that provides an optimized target in latent space, and learns both the mean and variance of the RL sampler. We further show that these two improvements are complementary, together yielding substantial gains over strong baselines. Empirically, our method improves over the strongest latent-reasoning baseline by +9.5% on FrozenLake spatial planning, with the gain widening to +19% at 32x32 grid map, and by +5.2% on average across nine visual-centric reasoning benchmarks.

Problem

• Latent Reasoning
• Supervised Fine-Tuning
• Reinforcement Learning
• Latent Representations

Innovation

• Scaffolding Minds
• latent visual target representations
• dedicated scaffolding encoder
• mean and variance of RL sampler
• multimodal reasoning