Learning the ARTS of Search for Automated Discovery

2026-06-20

AI Summary

Existing scientific discovery methods often conflate the quality of hypotheses with the effectiveness of experimental execution and prematurely prune search histories due to context length limitations, thereby overlooking high-quality initial hypotheses. This work proposes the ARTS framework, which for the first time decouples the evaluation of hypothesis value from execution quality by leveraging a reasoning language model to analyze execution logs and distinguish the root causes of failure, thereby guiding subsequent exploration. Additionally, ARTS introduces a test-time training mechanism that compresses knowledge from the search tree into model parameters, circumventing context window constraints. Evaluated on 22 tasks across MLGym and MLEBench, ARTS achieves a normalized score improvement of over 15.3%. A fine-tuned Qwen3-4B model under this framework matches the performance of Gemini-3 Pro and GPT-o3-reasoning at one-fifth the inference cost and demonstrates superior results on reinforcement learning tasks.

Abstract

Scientific discovery can be formulated as an iterative search process over the space of hypotheses and experiments. Contemporary methods navigate this space using heuristics such as MCTS. These algorithms conflate the merit of a hypothesis with the quality of its experimental execution. A promising hypothesis with preliminary execution is therefore ranked below a modest hypothesis whose execution is refined. Moreover, prior methods prune the search logs as the search progresses because the accumulated history outgrows the context window. We propose Agentic Reasoning for Tree Search (ARTS), where we deploy a reasoning language model to navigate this space. The model inspects prior execution logs, diagnoses whether earlier failures arose from faulty implementations or bad hypotheses, and selects the hypothesis to build on next. To mitigate challenges with context length, ARTS uses test-time training to instill the knowledge of search tree in the model weights. Across 22 tasks from MLGym and MLEBench, we show that ARTS outperforms leading algorithms, with over 15.3% relative improvement in the normalized score. With test-time training we show that a Qwen3-4B agent can match performance with closed-source frontier models like Gemini-3 Pro and GPT o3-reasoning with upto 5x lower inference cost. We further observe that on partially observable RL tasks, the test-time trained Qwen3-4B scientist surpasses ARTS with the o3 scientist by rediscovering the human-best recurrent-memory solution that heuristic methods prune away.

Problem

• scientific discovery
• hypothesis evaluation
• search space pruning
• context window limitation
• automated reasoning

Innovation

• Agentic Reasoning
• Tree Search
• Test-time Training
• Automated Scientific Discovery
• Reasoning Language Model