Graph-R1: Unleashing LLM Reasoning with NP-Hard Graph Problems

2025-08-27

AI Summary

To address the scalability challenges in training long-chain-of-thought (Long CoT) reasoning capabilities for large language models (LLMs) and the high cost of high-quality human-annotated data, this work introduces a synthetic data generation method grounded in NP-hard graph problems and a two-stage post-training framework. It is the first to leverage NP-hard graph problems as controllable, diverse, and infinitely scalable reasoning corpora, augmented by deep-reasoning guidance and reflective mechanisms to elicit extended reasoning chains. A fine-grained reward function is further designed, enabling joint optimization of reasoning path quality and efficiency via rejection-sampling–driven supervised fine-tuning and reinforcement learning. The resulting Graph-R1-7B model demonstrates strong generalization across mathematical, programming, STEM, and logical reasoning benchmarks, and notably outperforms QwQ-32B in both accuracy and inference efficiency on NP-hard graph problem solving.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Reasoning Large Language Models (RLLMs) have recently achieved remarkable progress on complex reasoning tasks, largely enabled by their long chain-of-thought (Long CoT) capabilities. However, developing these Long CoT behaviors relies heavily on post-training with high-quality datasets, which are typically costly and human-curated (e.g., mathematics and code), leaving scalable alternatives unexplored. In this work, we introduce NP-hard (NPH) graph problems as a novel synthetic training corpus, as they inherently require deep reasoning, extensive exploration, and reflective strategies, which are core characteristics of Long CoT reasoning. Building on this insight, we develop a two-stage post-training framework: (i) Long CoT Supervised Fine-Tuning (SFT) on rejection-sampled NPH graph instances, which substantially enhances reasoning depth, and (ii) Reinforcement Learning (RL) with a fine-grained reward design, which sharpens reasoning efficiency. Our flagship model, Graph-R1-7B, demonstrates strong generalization across mathematics, coding, STEM, and logic, and surpasses QwQ-32B on NPH graph problems in both accuracy and reasoning efficiency. These results position NPH graph problems as an effective and scalable resource for advancing Long CoT reasoning in LLMs, opening a new frontier for LLM post-training. Our implementation is available at https://github.com/Graph-Reasoner/Graph-R1, with models and datasets hosted in our Hugging Face collection HKUST-DSAIL/Graph-R1.

Problem

• Developing scalable alternatives to costly human-curated reasoning datasets
• Enhancing LLM reasoning depth and efficiency through graph problems
• Creating synthetic training corpus for long chain-of-thought capabilities

Innovation

• Synthetic NP-hard graph problems for scalable training
• Two-stage fine-tuning with SFT and reinforcement learning
• Fine-grained reward design to enhance reasoning efficiency