AI Can Learn Scientific Taste

2026-03-15

AI Summary

This work aims to enhance AI’s capacity to identify and generate high-impact scientific ideas by cultivating its “scientific taste.” To this end, it formalizes scientific taste as a preference modeling and alignment task and introduces a novel reinforcement learning paradigm based on community feedback (RLCF). The authors train a Scientific Judge reward model on 700,000 pairs of domain- and time-matched high- and low-citation papers, which is then used to optimize a Scientific Thinker for generating high-quality research ideas. Experimental results demonstrate that Scientific Judge outperforms existing large language models in evaluating scientific ideas and exhibits strong generalization across unseen domains, future time periods, and peer-review preferences. Moreover, the ideas generated by Scientific Thinker significantly surpass those produced by baseline methods.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Machine Learning: Learning Preferences or Rankings
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Great scientists have strong judgement and foresight, closely tied to what we call scientific taste. Here, we use the term to refer to the capacity to judge and propose research ideas with high potential impact. However, most relative research focuses on improving an AI scientist's executive capability, while enhancing an AI's scientific taste remains underexplored. In this work, we propose Reinforcement Learning from Community Feedback (RLCF), a training paradigm that uses large-scale community signals as supervision, and formulate scientific taste learning as a preference modeling and alignment problem. For preference modeling, we train Scientific Judge on 700K field- and time-matched pairs of high- vs. low-citation papers to judge ideas. For preference alignment, using Scientific Judge as a reward model, we train a policy model, Scientific Thinker, to propose research ideas with high potential impact. Experiments show Scientific Judge outperforms SOTA LLMs (e.g., GPT-5.2, Gemini 3 Pro) and generalizes to future-year test, unseen fields, and peer-review preference. Furthermore, Scientific Thinker proposes research ideas with higher potential impact than baselines. Our findings show that AI can learn scientific taste, marking a key step toward reaching human-level AI scientists.

Problem

• scientific taste
• AI scientist
• research impact
• preference modeling
• idea generation

Innovation

• scientific taste
• preference modeling
• reinforcement learning from community feedback
• impact prediction
• AI scientist