TreeThink: A Modular Tree Search Library for Mathematical Reasoning with LLMs

2026-07-13

AI Summary

This work addresses the lack of native support for formal verifiers in existing large language model (LLM) tree search libraries and the limited generality of theorem-proving systems, which often rely on task-specific search strategies that hinder unified, efficient cross-lingual mathematical reasoning. To bridge this gap, the authors propose a modular, fully asynchronous open-source tree search framework that, for the first time, natively integrates interactive theorem proving in Lean4, Coq (Rocq), and Isabelle/HOL. The framework unifies natural and formal language reasoning by coupling vLLM inference pipelines, lightweight heuristics, and neural evaluators through REPL interfaces. Experiments on the miniF2F and MATH500 benchmarks demonstrate that the approach effectively enables cross-lingual formal proof synthesis and achieves up to a 6.3× end-to-end speedup over prior methods.

Abstract

Tree search algorithms enable systematic exploration of the proof space in neural theorem proving. Existing LLM tree search libraries primarily target natural language reasoning and do not provide native integration with formal verifiers, while theorem proving systems often rely on task-specific search implementations. We introduce TreeThink, an open-source Python library for modular, fully asynchronous tree search in neural theorem proving. It integrates established tree search methods with vLLM-based inference pipelines and diverse node evaluation techniques, ranging from lightweight heuristics to neural evaluators. We support Lean~4, Rocq, and Isabelle/HOL alongside natural language. It connects directly to each language's Read-Eval-Print Loop (REPL) server for real-time verification and proof state extraction. We evaluate TreeThink on miniF2F and MATH500, demonstrating cross-language formal proof search, natural language reasoning support, and up to 6.3$\times$ wall-clock speedup from asynchronous execution. Source code is released under the MIT license at https://github.com/GGLAB-KU/treethink , and the library is accessible as a downloadable package at https://pypi.org/project/treethink/ .

Problem

• tree search
• neural theorem proving
• formal verification
• modular reasoning
• LLM-based reasoning

Innovation

• tree search
• neural theorem proving
• asynchronous execution
• formal verification
• modular library