QED: An Open-Source Multi-Agent System for Generating Mathematical Proofs on Open Problems

2026-04-27

AI Summary

Current large language models exhibit systematic failures in generating research-level mathematical proofs, often struggling to produce original and nontrivial solutions. To address this limitation, this work proposes QED, an open-source multi-agent system that systematically applies collaborative multi-agent mechanisms to solving open-ended mathematical problems. By integrating task decomposition, focused verification, hallucination-resistant citation, dynamic planning, and cross-model collaboration, QED effectively circumvents the bottlenecks of single-model approaches and mitigates context contamination. Evaluated on five expert-provided open problems, QED successfully generated three proofs that were verified as original, correct, and nontrivial, marking a significant leap from benchmark evaluation to real-world research scenarios.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Qualitative Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

We explore a central question in AI for mathematics: can AI systems produce original, nontrivial proofs for open research problems? Despite strong benchmark performance, producing genuinely novel proofs remains an outstanding challenge for LLMs. Through systematic experiments with frontier LLMs on research-level proof tasks, we identify seven failure modes that prevent reliable proof generation, including context contamination, citation hallucination, hand-waving on key steps and misallocation of proof effort, unstable proof plans, unfocused verification, problem modification and single-model bottleneck. We argue that the gap between benchmark success and research-level proving is primarily one of system design, due to those failure modes. We present QED, an open-source multi-agent proof system in which each architectural decision directly addresses a specific failure mode. Evaluated on five open problems in applied analysis and PDEs contributed by domain experts, QED produces correct proofs for three problems, each verified by the contributing experts as original and nontrivial. QED is released as open-source software at https://github.com/proofQED/QED.

Problem

• mathematical proofs
• open problems
• AI for mathematics
• large language models
• proof generation

Innovation

• multi-agent system
• mathematical proof generation
• open problems
• failure modes
• large language models