SciRisk-Bench: A Risk-Dimension-Aware Benchmark for AI4Science Safety

2026-06-17

AI Summary

This study addresses the lack of systematic characterization of domain-specific risks in current AI4Science safety evaluations, which hinders precise identification of model vulnerabilities in high-stakes scientific tasks. To bridge this gap, we propose the first dual-perspective benchmark that jointly considers risk dimensions and scientific disciplines, explicitly defining ten categories of scientific risks across seven major fields and 31 subdomains. Leveraging expert annotations from multiple disciplines and a diverse set of scientific tasks, we construct a structured evaluation dataset enabling fine-grained, cross-model, and cross-dimensional safety assessment. Empirical results reveal critical safety shortcomings of both general-purpose and science-specialized large language models across various risk types and disciplines, providing actionable diagnostic insights for future safety alignment efforts.

Abstract

Large language models (LLMs) are increasingly embedded in AI for Science (AI4Science) workflows, from scientific question answering and literature analysis to laboratory planning and autonomous discovery. This progress creates an urgent need for safety benchmarks that evaluate not only scientific competence, but also whether models recognize and avoid risks in high-stakes scientific contexts. Existing AI4Science safety datasets cover several disciplines and task formats, leaving the underlying risk dimensions underspecified. We introduce \textbf{SciRisk-Bench}, a benchmark designed to evaluate AI4Science safety from two complementary perspectives: explicit risk dimensions and scientific disciplines. SciRisk-Bench covers 7 disciplines, 31 subdisciplines and 10 risk dimensions. In the experimental section, we evaluate both mainstream LLMs and science-oriented LLMs across risk dimensions, disciplines, and sub-disciplines, enabling fine-grained diagnosis of where scientific models remain unsafe.

Problem

• AI4Science safety
• risk dimensions
• scientific disciplines
• safety benchmark
• large language models

Innovation

• risk-dimension-aware
• AI4Science safety
• scientific benchmark
• large language models
• fine-grained safety evaluation