Branch2Skill: Efficient Skill Evolution Through Reasoning Trees

2026-08-09

AI Summary

Existing skill evolution methods rely on a single reasoning trajectory, making them highly susceptible to early errors, which degrades feedback quality and incurs high iterative costs. This work proposes a reasoning-tree-based skill optimization framework that, within a fixed computational budget, leverages Monte Carlo Tree Search to generate diverse reasoning paths. By contrasting elite paths with their sibling paths sharing common prefixes, the method extracts multi-step reasoning evidence as dense supervision signals, enabling one-step joint updates across multiple reasoning steps. This approach is the first to transform a single reasoning tree into highly efficient supervisory information, substantially reducing redundant rollouts and token consumption. Evaluated across six reasoning and agent benchmarks, it consistently improves performance—outperforming SkillOpt while reducing token usage by 73.2% when using GPT-5.5.

Abstract

Skill evolution improves agent skills through feedback over time, with failed trajectories often providing informative signals by revealing incomplete or misleading behaviors. However, existing methods mainly rely on single trajectories, where early reasoning errors can propagate through subsequent steps and weaken the feedback available for skill refinement. Consequently, improving skills requires repeated cycles of rollout, diagnosis, and update, incurring substantial token costs. To address this challenge, we introduce Branch2Skill, an efficient framework that transforms a single reasoning tree into dense supervision for skill evolution. For each task or problem, Branch2Skill performs Monte Carlo tree search under a fixed budget to obtain diverse reasoning trajectories, then compares an elite path with sibling alternatives sharing the same prefixes to extract step-wise evidence about which reasoning patterns to retain, revise, or avoid. Finally, Branch2Skill distills multi-step evidence into reusable updates, allowing one reasoning tree to provide supervision across multiple reasoning steps and reducing the need for repeated rollout-update cycles. Across six benchmarks covering reasoning and agentic tasks, Branch2Skill consistently improves task performance while enhancing skill evolution efficiency. For example, with GPT 5.5 as the target model, Branch2Skill uses 73.2% fewer tokens than SkillOpt, while achieving superior performance. These results demonstrate that reasoning trees can support not only more effective trajectory search, but also richer supervision for more efficient skill improvement. Code will be published.

Problem

• skill evolution
• reasoning trajectories
• feedback propagation
• token efficiency
• trajectory diversity

Innovation

• reasoning trees
• skill evolution
• Monte Carlo tree search
• dense supervision
• trajectory comparison