Learning to Solve Complex Problems via Dataset Decomposition

2026-02-23

AI Summary

This work proposes a reverse curriculum learning framework tailored for structurally complex and conceptually deep tasks such as advanced mathematical problem solving and code generation. The approach introduces a difficulty scoring mechanism based on structural complexity and conceptual depth, and employs a teacher–student architecture to recursively decompose challenging problems. The teacher model generates progressively simplified examples through step-by-step reasoning, thereby constructing an easy-to-hard curriculum that guides the student model in incremental learning. Experimental results on benchmarks including MATH and AIME demonstrate that this method significantly outperforms standard training strategies, effectively enhancing the model’s capacity to solve complex problems.

Technology Category

• Machine Learning: Structured Learning
• Search and Optimization: Learning to Search
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Curriculum learning is a class of training strategies that organizes the data being exposed to a model by difficulty, gradually from simpler to more complex examples. This research explores a reverse curriculum generation approach that recursively decomposes complex datasets into simpler, more learnable components. We propose a teacher-student framework where the teacher is equipped with the ability to reason step-by-step, which is used to recursively generate easier versions of examples, enabling the student model to progressively master difficult tasks. We propose a novel scoring system to measure data difficulty based on its structural complexity and conceptual depth, allowing curriculum construction over decomposed data. Experiments on math datasets (MATH and AIME) and code generation datasets demonstrate that models trained with curricula generated by our approach exhibit superior performance compared to standard training on original datasets.

Problem

• curriculum learning
• dataset decomposition
• complex problem solving
• data difficulty
• structured reasoning

Innovation

• curriculum learning
• dataset decomposition
• teacher-student framework
• difficulty scoring
• recursive simplification