StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?

2026-09-01

AI Summary

研究提出StudyBench，一个物理基准测试，旨在评估自我进化方法从原始材料中学习并解决问题的能力，发现现有方法在难题迁移上表现不佳。

Abstract

Humans need to study only a handful of well-written textbooks to master a discipline and attempt its hardest problems. We argue that an ideal self-evolution method should share the same property, that is autonomously learning from raw training material for transferable problem-solving capability. However, we still lack a direct measurement for it. We introduce StudyBench, a controlled physics benchmark that directly measures how efficiently a self-evolution method converts training material into capability. We organise the test set into an Application Set, consisting of difficult textbook problems and evaluating absorption ability, and a Transfer Set, consisting of olympiad-level problems and evaluating transfer ability. Benchmarking representative self-evolution methods across three base models, we find that improvements on the Application Set rarely translate to the harder Transfer Set. A guidance ablation exposes a Guidance Gap: even the strongest method closes only a small fraction of what the same material unlocks when supplied as in-context guidance. Besides, every method hits a Compute Plateau, saturating well before exhausting its compute budget. The remaining gap is therefore a method problem rather than a data or compute problem. By offering a clean and controlled benchmark, StudyBench turns self-evolution progress from an open-ended pursuit into a measurable target for future research. Our code is released at https://github.com/thunlp/StudyBench.

Problem

• self-evolution
• transferable problem-solving capability
• direct measurement
• training material
• benchmark

Innovation

• self-evolution method
• transferable problem-solving capability
• Compute Plateau
• Guidance Gap
• StudyBench