Framework of Thoughts: A Foundation Framework for Dynamic and Optimized Reasoning based on Chains, Trees, and Graphs

2026-02-18

AI Summary

Existing reasoning methods often rely on static, task-specific structures that struggle to adapt to dynamic or previously unseen problems and exhibit limited optimization in hyperparameter configuration, prompt design, execution efficiency, and computational cost. To address these limitations, this work proposes FoT, a general-purpose framework that, for the first time, enables dynamic generation of reasoning structures—such as chains, trees, and graphs—and supports multi-dimensional automatic optimization. FoT integrates hyperparameter tuning, prompt engineering, parallel execution, and intelligent caching mechanisms. The framework significantly enhances reasoning efficiency, effectiveness, and generalizability, achieving faster execution, lower overhead, and superior task performance across multiple benchmarks. The code has been open-sourced to advance research in efficient and dynamic reasoning.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Search and Optimization: Metareasoning and Metaheuristics
• Natural Language Processing: Prompt Engineering / Prompting

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Prompting schemes such as Chain of Thought, Tree of Thoughts, and Graph of Thoughts can significantly enhance the reasoning capabilities of large language models. However, most existing schemes require users to define static, problem-specific reasoning structures that lack adaptability to dynamic or unseen problem types. Additionally, these schemes are often under-optimized in terms of hyperparameters, prompts, runtime, and prompting cost. To address these limitations, we introduce Framework of Thoughts (FoT)--a general-purpose foundation framework for building and optimizing dynamic reasoning schemes. FoT comes with built-in features for hyperparameter tuning, prompt optimization, parallel execution, and intelligent caching, unlocking the latent performance potential of reasoning schemes. We demonstrate FoT's capabilities by implementing three popular schemes--Tree of Thoughts, Graph of Thoughts, and ProbTree--within FoT. We empirically show that FoT enables significantly faster execution, reduces costs, and achieves better task scores through optimization. We release our codebase to facilitate the development of future dynamic and efficient reasoning schemes.

Problem

• reasoning schemes
• dynamic reasoning
• prompt optimization
• large language models
• adaptive structures

Innovation

• Framework of Thoughts
• dynamic reasoning
• prompt optimization
• parallel execution
• intelligent caching