On the Diagram of Thought

2024-09-16

arXiv.org

AI Summary

This work addresses the key limitation in large language models (LLMs) where iterative reasoning is constrained to linear chains or trees, hindering simultaneous support for diverse reasoning paths and logical consistency. To this end, we propose the Diagram of Thought (DoT) framework, which unifies iterative reasoning within a single model via a directed acyclic graph (DAG)—where nodes represent propositions alongside their critique, refinement, and verification—and enables natural-language-feedback-driven self-optimization. Methodologically, DoT introduces a novel formal foundation grounded in Topos theory and a role-specific token-augmented autoregressive prediction mechanism, allowing seamless alternation between generative and critical roles within one model. Experiments demonstrate that DoT significantly improves both reasoning consistency and path exploration capability—without requiring multi-model collaboration or external controllers—establishing a new paradigm for scalable, principle-driven design of reasoning-specialized models.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

We introduce Diagram of Thought (DoT), a framework that models iterative reasoning in large language models (LLMs) as the construction of a directed acyclic graph (DAG) within a single model. Unlike conventional approaches that represent reasoning as linear chains or tree structures, DoT organizes propositions, critiques, refinements, and verifications into a unified DAG, enabling the exploration of complex reasoning pathways while preserving logical consistency. In this framework, each node encapsulates a proposition at various stages of evaluation, thereby facilitating iterative self-improvement through detailed natural language feedback. By leveraging auto-regressive next-token prediction augmented with role-specific tokens, DoT seamlessly transitions between generating ideas and engaging in critical evaluation, offering richer, context-aware feedback than binary signals. Moreover, we establish a rigorous mathematical foundation for DoT through Topos Theory, ensuring soundness and consistency in the reasoning process. This integrated approach not only simplifies both training and inference by eliminating the need for multiple models or external control mechanisms but also provides a principled framework for the design of next-generation reasoning-specialized models.

Problem

• Models iterative reasoning in LLMs using a DAG structure.
• Enables complex reasoning pathways with logical consistency.
• Simplifies training and inference by eliminating multiple models.

Innovation

• Models reasoning as a directed acyclic graph (DAG).
• Uses auto-regressive prediction with role-specific tokens.
• Establishes mathematical foundation via Topos Theory.