Interaction Scaling: Grounding the Third Axis of Test-Time Compute

2026-07-13

AI Summary

Current test-time computation methods—such as extended inference or multi-round sampling—are constrained by the model’s internal knowledge and struggle to incorporate new information to overcome performance bottlenecks. This work proposes interaction as a novel dimension of test-time computation, introducing a closed-loop iterative framework in which the model generates outputs, external tools perform real-world observations, and grounded feedback drives continuous refinement. The study provides the first systematic definition and empirical validation of interaction scaling, emphasizing that effective feedback must derive from real-world measurements rather than model self-evaluation alone. Under a fixed computational budget, this approach achieves a 100% variance-free pass rate on code generation tasks and reduces defects by 40–74% in visual layout tasks, substantially outperforming baselines that rely solely on vision-language model scoring.

Abstract

There are two standard ways to spend more compute at test time: let a model reason longer, or sample more attempts and keep one. Both share a hidden limit: they are internal. Every extra token comes from the same frozen weights and the same prompt, so neither can tell the model anything it does not already know. We study a third way, interaction: the model proposes an artifact, an external instrument observes how it actually behaves, and the model revises. Each cycle imports a real observation, so interaction breaks through the ceiling the other two hit. We argue that a single variable governs this third axis, grounding, and that it must hold on both sides of the loop. The feedback that drives revision must come from an instrument that actually observes the flaw, and so must the metric that scores the result. On hard coding tasks at a fixed token budget, reasoning-only and best-of-N sampling both plateau (the latter even when an oracle picks the best sample), while every interaction strategy keeps improving; our proposer-reviewer harness reaches a perfect 100% pass rate with no run-to-run variance, and the gain holds across three model families. On rendered visual artifacts, the usual judge (a vision-language model, or VLM, reading a screenshot) rates 14 of 15 visibly broken figures "perfect," because the screenshot hides the flaws before the judge can see them. A tool that measures the real layout instead shows the loop removing 40-74% of defects across four modalities; and that same VLM, used as the reviewer, makes slide layouts worse where the measuring tool repairs them. Interaction scaling is real and distinct from reasoning and sampling, but only visible when both the feedback and the metric are grounded.

Problem

• interaction scaling
• test-time compute
• grounding
• external feedback
• model revision

Innovation

• Interaction Scaling
• Grounding
• Test-Time Compute
• External Feedback
• Proposer-Reviewer Loop