🤖 AI Summary
This work addresses the limitation of traditional Graph of Thoughts frameworks, which rely on manually predefined operation graphs and struggle to adapt to varying task complexity. To overcome this, the paper introduces a novel approach that integrates large language models with reinforcement learning to automatically construct task-adaptive reasoning operation graphs from a human-defined set of operations. By leveraging reinforcement learning, the method dynamically generates operation graphs tailored to the specific demands of each task, moving beyond the constraints of static, handcrafted designs. This enables adaptive graph construction under given constraints, significantly enhancing both the flexibility and performance of complex problem-solving. The proposed framework represents the first application of reinforcement learning within the Graph of Thoughts paradigm, offering a principled and scalable solution for automating reasoning structures.
📝 Abstract
Graph of Thoughts (GoT), a generalized form of recent prompting paradigms for large language models (LLMs), has been shown to be useful for elaborate problem solving. By executing a graph of operations, thoughts of the LLM are structured as an arbitrary graph, forming the actual graph of thoughts. Originally, the graph of operations is defined manually, which requires in-depth knowledge about the solution of the problem to solve. Such a static graph of operations is rigid and therefore lacks adaptability. We propose Reinforced Graph of Thoughts (RGoT), an automated approach to the GoT prompting paradigm that leverages reinforcement learning (RL) to adaptively generate a graph of operations from a human-defined set. Results indicate that, under certain constraints, it is possible to construct graphs of operations adaptively to the task's complexity in an automated way.