Toward Template-Free Explainability for Monte Carlo Tree Search

2026-05-15

AI Summary

This work addresses the challenge of explaining decisions made by Monte Carlo Tree Search (MCTS), which are often opaque due to its asymmetric tree structure and simulation-based value estimation. To this end, the authors propose an end-to-end interpretability method that leverages a large language model (LLM) to generate evidence-based natural language explanations directly from MCTS search trajectories. The approach first classifies user intent to understand the query, then dynamically evaluates the sufficiency of evidence within the search tree and triggers targeted expansions when necessary. Explanations are synthesized using visit counts, value estimates, and risk-aware information. This study presents the first framework capable of producing adaptive, evidence-grounded explanations for probabilistic search algorithms without relying on handcrafted templates or intermediate formal representations, demonstrating that LLMs can effectively serve as end-to-end interpreters for MCTS.

Abstract

Probabilistic search algorithms, such as Monte Carlo Tree Search (MCTS), have proven very effective in solving sequential decision-making tasks under uncertainty. However, interpreting asymmetric search trees that incorporate bandit-based tree traversal and simulation-based value estimation is difficult for end users based solely on raw tree statistics. While prior work requires hand-crafted formal logic constraints that must be updated when the problem changes, we present a framework that enables large language models (LLMs) to generate evidence-grounded explanations of MCTS decisions from recorded search traces in an end-to-end manner. Our framework maps natural-language questions to a structured set of intent categories, determines whether the existing tree contains sufficient evidence, triggers targeted expansion when needed, and generates explanations using tree statistics such as visit counts, value estimates, and risk information. Experimental results provide the first evidence that LLMs can serve as end-to-end explainers for probabilistic search, without requiring intermediate formal representations.

Problem

• Explainability
• Monte Carlo Tree Search
• Template-Free
• Probabilistic Search
• Decision-Making

Innovation

• Monte Carlo Tree Search
• Large Language Models
• Explainable AI
• Template-Free Explanation
• Search Trace Interpretation