Biosecurity-Aware AI: Agentic Risk Auditing of Soft Prompt Attacks on ESM-Based Variant Predictors

2025-12-18

AI Summary

While genomic foundation models (e.g., ESM2) achieve strong performance in variant effect prediction, their robustness against soft-prompt adversarial attacks remains systematically unassessed. Method: We propose Secure Agentic Genomic Evaluator (SAGE), an active, model-agnostic risk auditing framework that leverages explainable agent-based reasoning loops—without modifying the target model—to enable automated, narrative-driven security evaluation for genomic AI. SAGE integrates soft-prompt adversarial perturbations, multi-checkpoint behavioral monitoring, AUROC/AUPR-based quantitative robustness analysis, and LLM-guided structured reporting to enable continuous auditing of embedding-space robustness. Contribution/Results: Empirical evaluation reveals significant soft-prompt vulnerabilities in ESM2 on clinical variant interpretation tasks. SAGE consistently detects latent security flaws with high fidelity, establishing the first deployable, bioinformatics-specific security auditing tool for biomedical AI models.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Adversarial Learning & Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Genomic Foundation Models (GFMs), such as Evolutionary Scale Modeling (ESM), have demonstrated remarkable success in variant effect prediction. However, their security and robustness under adversarial manipulation remain largely unexplored. To address this gap, we introduce the Secure Agentic Genomic Evaluator (SAGE), an agentic framework for auditing the adversarial vulnerabilities of GFMs. SAGE functions through an interpretable and automated risk auditing loop. It injects soft prompt perturbations, monitors model behavior across training checkpoints, computes risk metrics such as AUROC and AUPR, and generates structured reports with large language model-based narrative explanations. This agentic process enables continuous evaluation of embedding-space robustness without modifying the underlying model. Using SAGE, we find that even state-of-the-art GFMs like ESM2 are sensitive to targeted soft prompt attacks, resulting in measurable performance degradation. These findings reveal critical and previously hidden vulnerabilities in genomic foundation models, showing the importance of agentic risk auditing in securing biomedical applications such as clinical variant interpretation.

Problem

• Auditing adversarial vulnerabilities of Genomic Foundation Models.
• Assessing security and robustness under soft prompt attacks.
• Evaluating embedding-space robustness without modifying the model.

Innovation

• Agentic framework audits genomic model vulnerabilities
• Soft prompt perturbations test embedding-space robustness
• Automated loop generates risk metrics and explanations