Problem
Research questions and friction points this paper is trying to address.
Retrieval
Ideation Operations
Scientific Literature
Inspiration
RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature
🤖 AI Summary
研究通过创建RATIO基准，利用特定类型的思想操作（解决、扩展、具体化）来改进科学文献中的信息检索方法，支持基于文献的创新思考。
📝 Abstract
Retrieved scientific literature can serve as inspiration for both human and AI scientists. Inspiration can take different forms: prior work may directly suggest how to address a problem, or surface directions at different levels of abstraction - zooming out to a more general view or zooming in to a concrete realization. We introduce RATIO (Retrieval Across Typed Ideation Operations), a large-scale benchmark in which relevance is defined by three operations which we name ideation moves: Address retrieves potential approaches for stated problems, Broaden retrieves more general formulations, and Specify retrieves concrete instantiations. RATIO is constructed from millions of full-text scientific papers across CS literature via a general recipe that extends discourse-marker distant supervision - previously used only for classification - to corpus-scale retrieval, combined with extensive LLM and human vetting. Experiments show that operation-specific fine-tuning substantially boosts retrievers but leaves much room for further improvements. RATIO provides a scalable training and evaluation framework for retrieval components that support literature-grounded ideation, opening up new research avenues on scientific inspiration retrieval.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Retrieval Across Typed Ideation Operations
ideation moves
discourse-marker distant supervision
LLM and human vetting
operation-specific fine-tuning
🔎 Similar Papers
