Recursive Decomposition with Dependencies for Generic Divide-and-Conquer Reasoning

2025-05-05

AI Summary

Large language models (LLMs) suffer from poor scalability, low inference efficiency, and heavy reliance on abundant task-specific supervision for complex reasoning tasks. Method: This paper proposes Recursive Decomposition and Dependency (RDD), a general divide-and-conquer framework that operates without task-specific examples. RDD integrates explicit subtask dependency modeling and an automatic backtracking-based error recovery mechanism into an unsupervised reasoning pipeline, enabling end-to-end problem solving via LLM-driven recursive decomposition, dependency graph construction, and ordered subtask scheduling. Contribution/Results: Evaluated across two benchmarks spanning six difficulty levels, RDD significantly outperforms chain-of-thought and other baselines under identical computational budgets—especially on high-complexity tasks—while simultaneously improving inference efficiency and cross-task generalization.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Reasoning tasks are crucial in many domains, especially in science and engineering. Although large language models (LLMs) have made progress in reasoning tasks using techniques such as chain-of-thought and least-to-most prompting, these approaches still do not effectively scale to complex problems in either their performance or execution time. Moreover, they often require additional supervision for each new task, such as in-context examples. In this work, we introduce Recursive Decomposition with Dependencies (RDD), a scalable divide-and-conquer method for solving reasoning problems that requires less supervision than prior approaches. Our method can be directly applied to a new problem class even in the absence of any task-specific guidance. Furthermore, RDD supports sub-task dependencies, allowing for ordered execution of sub-tasks, as well as an error recovery mechanism that can correct mistakes made in previous steps. We evaluate our approach on two benchmarks with six difficulty levels each and in two in-context settings: one with task-specific examples and one without. Our results demonstrate that RDD outperforms other methods in a compute-matched setting as task complexity increases, while also being more computationally efficient.

Problem

• Scalable divide-and-conquer reasoning for complex problems
• Reducing supervision needs in task-specific reasoning
• Handling sub-task dependencies and error recovery

Innovation

• Recursive Decomposition with Dependencies for reasoning
• Scalable divide-and-conquer method with less supervision
• Supports sub-task dependencies and error recovery