T$^2$: An Adaptive Test-Time Scaling Strategy for Contextual Question Answering

2025-05-23

AI Summary

Large language models (LLMs) in conversational question answering (CQA) suffer from poor adaptability due to fixed inference depth, while existing test-time adaptation methods introduce human-induced bias. Method: We propose T², a dynamic inference-depth control framework centered on the novel “Think-to-Think” mechanism. It automatically assesses question complexity via structural parsing, augments prompts with semantically similar examples, and selects and transfers inference strategies through multi-criteria evaluation—namely consistency, conciseness, and verifiability—ensuring unbiased, adaptive reasoning. Contribution/Results: T² eliminates reliance on manual heuristics and premature termination constraints. Evaluated on seven CQA benchmarks, it achieves average accuracy gains over strong baselines while reducing computational overhead by up to 25.2%, demonstrating superior trade-offs between performance and inference efficiency.

Technology Category

• Natural Language Processing: Question Answering
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Recent advances in Large Language Models (LLMs) have demonstrated remarkable performance in Contextual Question Answering (CQA). However, prior approaches typically employ elaborate reasoning strategies regardless of question complexity, leading to low adaptability. Recent efficient test-time scaling methods introduce budget constraints or early stop mechanisms to avoid overthinking for straightforward questions. But they add human bias to the reasoning process and fail to leverage models' inherent reasoning capabilities. To address these limitations, we present T$^2$: Think-to-Think, a novel framework that dynamically adapts reasoning depth based on question complexity. T$^2$ leverages the insight that if an LLM can effectively solve similar questions using specific reasoning strategies, it can apply the same strategy to the original question. This insight enables to adoption of concise reasoning for straightforward questions while maintaining detailed analysis for complex problems. T$^2$ works through four key steps: decomposing questions into structural elements, generating similar examples with candidate reasoning strategies, evaluating these strategies against multiple criteria, and applying the most appropriate strategy to the original question. Experimental evaluation across seven diverse CQA benchmarks demonstrates that T$^2$ not only achieves higher accuracy than baseline methods but also reduces computational overhead by up to 25.2%.

Problem

• Adapts reasoning depth based on question complexity
• Avoids human bias in reasoning strategies
• Reduces computational overhead while improving accuracy

Innovation

• Dynamic reasoning depth adaptation
• Similar example strategy evaluation
• Reduced computational overhead significantly