AI-Assisted Scientific Assessment: A Case Study on Climate Change

2026-02-10

AI Summary

This study addresses complex scientific assessment tasks—such as evaluating the stability of the Atlantic Meridional Overturning Circulation (AMOC) in climate science—that lack reproducible validation and rely heavily on expert consensus. For the first time, the authors integrate a Gemini large language model–based AI collaboration system into standard scientific workflows to support literature synthesis, logical consistency maintenance, and report generation. This approach transcends traditional trial-and-error paradigms of AI-assisted research by enabling automated support for high-level theoretical reasoning and evidence integration. In practice, 13 scientists synthesized findings from 79 papers within 46 person-hours across 104 revision cycles. While most AI-generated content was retained, rigorous expert review remained essential to uphold scientific rigor and validity.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Humans and AI: Human-AI Collaboration / Human-AI Teaming

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The emerging paradigm of AI co-scientists focuses on tasks characterized by repeatable verification, where agents explore search spaces in'guess and check'loops. This paradigm does not extend to problems where repeated evaluation is impossible and ground truth is established by the consensus synthesis of theory and existing evidence. We evaluate a Gemini-based AI environment designed to support collaborative scientific assessment, integrated into a standard scientific workflow. In collaboration with a diverse group of 13 scientists working in the field of climate science, we tested the system on a complex topic: the stability of the Atlantic Meridional Overturning Circulation (AMOC). Our results show that AI can accelerate the scientific workflow. The group produced a comprehensive synthesis of 79 papers through 104 revision cycles in just over 46 person-hours. AI contribution was significant: most AI-generated content was retained in the report. AI also helped maintain logical consistency and presentation quality. However, expert additions were crucial to ensure its acceptability: less than half of the report was produced by AI. Furthermore, substantial oversight was required to expand and elevate the content to rigorous scientific standards.

Problem

• AI-assisted scientific assessment
• scientific consensus
• climate change
• AMOC stability
• collaborative synthesis

Innovation

• AI-assisted scientific assessment
• collaborative AI
• scientific workflow integration
• consensus synthesis
• Atlantic Meridional Overturning Circulation (AMOC)