Extending Test-Time Scaling: A 3D Perspective with Context, Batch, and Turn

2025-11-18

AI Summary

Standard inference for foundation models is constrained by fixed context length, limiting reasoning over long or interactive sequences. Method: This paper proposes 3D-TIME—a three-dimensional test-time scaling framework that jointly expands inference capacity along context length, batch size, and reasoning iterations. It introduces batch scaling and iterative self-refinement to test-time inference for the first time, unifying context augmentation, parallel sampling, and iterative refinement. The framework supports human-in-the-loop feedback and embodied learning. A reinforcement learning–based reasoning model integrates multi-path sampling, dynamic context-length expansion, and preference modeling. Results: 3D-TIME achieves significant accuracy gains on complex reasoning benchmarks—including IOI, IMO, and CPHO—demonstrating scalability across all three dimensions. Incorporating human feedback further improves performance, establishing a novel paradigm for long-horizon, interactive reasoning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Reasoning reinforcement learning (RL) has recently revealed a new scaling effect: test-time scaling. Thinking models such as R1 and o1 improve their reasoning accuracy at test time as the length of the reasoning context increases. However, compared with training-time scaling, test-time scaling is fundamentally limited by the limited context length of base models, which remains orders of magnitude smaller than the amount of tokens consumed during training. We revisit test-time enhancement techniques through the lens of scaling effect and introduce a unified framework of multi-dimensional test-time scaling to extend the capacity of test-time reasoning. Beyond conventional context-length scaling, we consider two additional dimensions: batch scaling, where accuracy improves with parallel sampling, and turn scaling, where iterative self-refinement enhances reasoning quality. Building on this perspective, we propose 3D test-time scaling, which integrates context, batch, and turn scaling. We show that: (1) each dimension demonstrates a test-time scaling effect, but with a bounded capacity; (2) combining all three dimensions substantially improves the reasoning performance of challenging testbeds, including IOI, IMO, and CPHO, and further benefits from human preference feedback; and (3) the human-in-the-loop framework naturally extends to a more open-ended domain, i.e., embodied learning, which enables the design of humanoid control behaviors.

Problem

• Extending test-time scaling beyond context length limitations
• Integrating batch and turn scaling to enhance reasoning quality
• Improving performance on challenging benchmarks through multi-dimensional scaling

Innovation

• Integrates context batch turn scaling dimensions
• Enhances reasoning via parallel sampling iterations
• Combines scaling dimensions for bounded capacity improvement