Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models

2025-10-31

AI Summary

Open-weight foundation models in biology pose dual-use risks: while accelerating scientific discovery and drug development, they may also be misused for bioweapon design. Current mitigation strategies rely primarily on filtering biohazardous data during pretraining, yet their robustness against malicious fine-tuning remains unassessed. This work introduces the first systematic evaluation framework targeting three critical capabilities—viral sequence modeling, mutational effect prediction, and virulence forecasting—employing fine-grained task benchmarks, adversarial fine-tuning experiments, and linear probe analysis. Results demonstrate that: (1) filtered hazardous knowledge is readily recoverable via fine-tuning; (2) dual-use signals are linearly decodable from model representations; and (3) existing data filtering fails to prevent targeted misuse. These findings expose fundamental limitations of current safety mechanisms and provide empirical evidence and methodological foundations for developing more robust governance strategies for open biological foundation models.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Open-weight bio-foundation models present a dual-use dilemma. While holding great promise for accelerating scientific research and drug development, they could also enable bad actors to develop more deadly bioweapons. To mitigate the risk posed by these models, current approaches focus on filtering biohazardous data during pre-training. However, the effectiveness of such an approach remains unclear, particularly against determined actors who might fine-tune these models for malicious use. To address this gap, we propose eval, a framework to evaluate the robustness of procedures that are intended to reduce the dual-use capabilities of bio-foundation models. eval assesses models' virus understanding through three lenses, including sequence modeling, mutational effects prediction, and virulence prediction. Our results show that current filtering practices may not be particularly effective: Excluded knowledge can be rapidly recovered in some cases via fine-tuning, and exhibits broader generalizability in sequence modeling. Furthermore, dual-use signals may already reside in the pretrained representations, and can be elicited via simple linear probing. These findings highlight the challenges of data filtering as a standalone procedure, underscoring the need for further research into robust safety and security strategies for open-weight bio-foundation models.

Problem

• Evaluating dual-use risks of open-weight bio-foundation models
• Assessing robustness of safety procedures against malicious fine-tuning
• Testing effectiveness of data filtering for biosecurity mitigation

Innovation

• Framework evaluates dual-use robustness of bio-foundation models
• Assesses virus understanding through three prediction lenses
• Reveals limitations of data filtering via fine-tuning experiments