Tree-of-Concerns: Hierarchical Multi-Agent Debate for Unstated-Limitation Extraction in Scientific Critique

2026-08-21

AI Summary

为解决科学论文中未报告的局限性问题，本文提出Tree-of-Concerns框架，通过多代理辩论机制提取这些隐含的失败模式。

Abstract

As scientific literature grows and papers increasingly under-report limitations, multi-agent LLMs offer a promising approach to systematically uncover these hidden failure modes. Here, we introduce Tree-of-Concerns, a multi-agent framework that deploys specialized skeptic personas, each operating through a category-specific analytical lens, as parallel debate trees to extract unstated limitations from scientific papers. Each persona conducts structured, evidence-grounded argumentation, while a Panel Review mechanism re-evaluates each surviving claim from all five perspectives to correct category drift and severity miscalibration. Through experiments on ToC-Bench, our benchmark of 414 research papers with 1,905 unstated limitations, sourced from reviewer-reported weaknesses and follow-up citation critiques, we demonstrate that ToC improves precision by 79% and coverage by 11% relative to strongest baselines, surfacing specific, evidence-grounded concerns that support reviewers in systematic evaluation.

Problem

• Scientific Critique
• Unstated Limitations
• Multi-Agent LLMs

Innovation

• multi-agent framework
• Tree-of-Concerns
• unstated limitations
• Panel Review mechanism
• evidence-grounded argumentation