AdaTurn: Budget-Aware Test-Time Scaling for Active Visual Perception Agents

2026-07-16

AI Summary

This work addresses the “catastrophic truncation” problem in active visual perception agents, where premature termination under limited reasoning steps prevents the generation of valid answers. To mitigate this, the authors propose a budget-aware framework that treats the step budget as a conditional input and introduces Forced-Answer DAPO (FA-DAPO), an algorithm that reformulates out-of-budget scenarios into trainable final decision steps. The approach further incorporates budget randomization during training and a load-balancing scheduling mechanism to explicitly optimize boundary behaviors. Experiments demonstrate that under a four-step constraint, the method boosts VisualProbe-Medium’s accuracy from 36.7% to 47.6%, maintains strong scalability under higher budgets, and generalizes effectively across diverse backbone architectures and multimodal benchmarks.

Abstract

Active visual agents solve fine-grained image tasks by interleaving reasoning with image-grounding actions across multiple turns. However, deployment-time rollout budgets are rarely fixed: some requests permit long rollouts, while others require the agent to act under a tight turn limit. Existing methods train the policy as if the rollout budget were hidden, so when the available budget is smaller than the trajectory the agent prefers, the interaction is often truncated before any valid answer is produced; we term this failure \emph{catastrophic truncation}. To overcome this challenge, we present AdaTurn, a budget-aware framework that conditions the agent on the allowed number of turns and explicitly trains the boundary behavior induced by the budget. Our key component, Forced-Answer DAPO (FA-DAPO), converts the over-budget event from a masked or penalized failure into a trainable final-decision step, teaching the model to synthesize partial evidence when further tool use is no longer possible. We further randomize rollout budgets during both training and inference and introduce a load-balanced scheduler that makes such operations practical. AdaTurn substantially improves low-budget accuracy, for example raising VisualProbe-Medium from 36.7% to 47.6% at four turns, while preserving strong scaling at larger budgets and transferring effectively to multiple backbones and general multimodal benchmarks.

Problem

• active visual perception
• test-time scaling
• rollout budget
• catastrophic truncation
• turn-limited interaction

Innovation

• budget-aware
• test-time scaling
• active visual perception
• forced-answer policy
• catastrophic truncation