Institution profile

Center for AI Safety

Academic institutionnorthamerica · us
Official website
Research library23linked papers
Opportunities0open roles
Selected work

Representative Papers

Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark

Apr 21, 2025

Assessing large language models’ (LLMs) capability to debug high-consequence virology experimental protocols—and their potential role in dual-use technology governance—remains unexplored. Method: We introduce VCT, the first multimodal question-answering benchmark for virology, comprising 322 expert-crafted questions spanning foundational, tacit, and visual knowledge domains; we further propose a multimodal LLM evaluation framework integrating text, flowchart, and experimental image understanding. Contribution/Results: This work presents the first systematic quantification of LLM performance on dual-use virological procedural reasoning: the o3 model achieves 43.8% accuracy—significantly exceeding both the human expert average (22.1%) and 94% of individual experts. These findings provide empirical evidence and a methodological foundation for integrating LLMs into life science dual-use governance frameworks.

1 citationsRead paper

Reducing Political Manipulation with Consistency Training

May 21, 2026

This study addresses the systematic and asymmetric implicit political bias exhibited by large language models in sensitive political contexts, which leads to imbalanced treatment of ideologically opposing viewpoints. The work introduces the first formal definition and quantification of seven distinct mechanisms of implicit political bias and proposes a Political Consistency Training (PCT) framework. PCT employs reinforcement learning to jointly optimize affective consistency and helpfulness consistency, thereby achieving symmetric rhetorical expression and interaction depth across politically opposing prompts. Experimental results demonstrate that this approach significantly reduces implicit bias while maintaining strong generalization across multiple held-out evaluation benchmarks and preserving overall model utility.

0 citationsRead paper

Protecting and Promoting Human Agency in Education in the Age of Artificial Intelligence

Feb 23, 2026

The widespread integration of generative artificial intelligence in education poses a potential threat to human agency, necessitating a systematic response. This study addresses this challenge through an interdisciplinary conceptual analysis, proposing a four-dimensional framework of human agency encompassing human oversight, human–AI complementarity, AI literacy, and relational emergence. The framework elucidates the critical tension between ethical considerations and operational efficacy in AI-enabled educational practices, particularly highlighting practical dilemmas related to normative constraints, transparency, and cognitive offloading. Designed to be technology-agnostic, this approach offers both a theoretical foundation and actionable guidance for the responsible and effective incorporation of generative AI in educational contexts.

0 citationsRead paper

Aggressive Compression Enables LLM Weight Theft

Jan 03, 2026arXiv.org

This work addresses the vulnerability of large language model (LLM) weights to theft, highlighting that their high compressibility significantly exacerbates leakage risks. For the first time, it directly links model compressibility to weight extraction in adversarial settings and proposes an aggressive compression method tailored for attack scenarios. By relaxing decompression constraints, this approach achieves compression ratios of 16–100×, reducing illicit transmission time from months to days. To counter this threat, the study introduces a low-cost, efficient forensic watermarking mechanism integrated with model obfuscation and hidden localization techniques, forming three complementary defense strategies. Experimental results demonstrate that the proposed watermarking scheme substantially enhances traceability while preserving model utility, offering a novel pathway for protecting intellectual property in LLMs.

0 citationsRead paper

Depth-Wise Activation Steering for Honest Language Models

Dec 08, 2025

Large language models (LLMs) often generate factually incorrect statements despite possessing correct knowledge—a failure of honesty rather than factual accuracy—undermining auditability and safety. To address this, we propose a training-free activation steering method that introduces a depth-aware Gaussian scheduling mechanism to dynamically allocate intervention strength across network layers, enabling weighted modulation of hidden-layer representations. Unlike uniform, random, or single-layer interventions, our approach more effectively elicits intrinsic honesty without fine-tuning and maintains cross-architectural compatibility. On the MASK benchmark, it significantly improves honesty in six out of seven mainstream LLMs. Ablation studies confirm that Gaussian scheduling uniquely disentangles honesty from knowledge retention, outperforming alternative scheduling strategies. This work advances controllable honesty in LLMs through interpretable, parameter-free intervention.

0 citationsRead paper
Recent publications

Latest Papers

Reducing Political Manipulation with Consistency Training

May 21, 2026

This study addresses the systematic and asymmetric implicit political bias exhibited by large language models in sensitive political contexts, which leads to imbalanced treatment of ideologically opposing viewpoints. The work introduces the first formal definition and quantification of seven distinct mechanisms of implicit political bias and proposes a Political Consistency Training (PCT) framework. PCT employs reinforcement learning to jointly optimize affective consistency and helpfulness consistency, thereby achieving symmetric rhetorical expression and interaction depth across politically opposing prompts. Experimental results demonstrate that this approach significantly reduces implicit bias while maintaining strong generalization across multiple held-out evaluation benchmarks and preserving overall model utility.

0 citationsRead paper

Protecting and Promoting Human Agency in Education in the Age of Artificial Intelligence

Feb 23, 2026

The widespread integration of generative artificial intelligence in education poses a potential threat to human agency, necessitating a systematic response. This study addresses this challenge through an interdisciplinary conceptual analysis, proposing a four-dimensional framework of human agency encompassing human oversight, human–AI complementarity, AI literacy, and relational emergence. The framework elucidates the critical tension between ethical considerations and operational efficacy in AI-enabled educational practices, particularly highlighting practical dilemmas related to normative constraints, transparency, and cognitive offloading. Designed to be technology-agnostic, this approach offers both a theoretical foundation and actionable guidance for the responsible and effective incorporation of generative AI in educational contexts.

0 citationsRead paper

Aggressive Compression Enables LLM Weight Theft

Jan 03, 2026arXiv.org

This work addresses the vulnerability of large language model (LLM) weights to theft, highlighting that their high compressibility significantly exacerbates leakage risks. For the first time, it directly links model compressibility to weight extraction in adversarial settings and proposes an aggressive compression method tailored for attack scenarios. By relaxing decompression constraints, this approach achieves compression ratios of 16–100×, reducing illicit transmission time from months to days. To counter this threat, the study introduces a low-cost, efficient forensic watermarking mechanism integrated with model obfuscation and hidden localization techniques, forming three complementary defense strategies. Experimental results demonstrate that the proposed watermarking scheme substantially enhances traceability while preserving model utility, offering a novel pathway for protecting intellectual property in LLMs.

0 citationsRead paper

Depth-Wise Activation Steering for Honest Language Models

Dec 08, 2025

Large language models (LLMs) often generate factually incorrect statements despite possessing correct knowledge—a failure of honesty rather than factual accuracy—undermining auditability and safety. To address this, we propose a training-free activation steering method that introduces a depth-aware Gaussian scheduling mechanism to dynamically allocate intervention strength across network layers, enabling weighted modulation of hidden-layer representations. Unlike uniform, random, or single-layer interventions, our approach more effectively elicits intrinsic honesty without fine-tuning and maintains cross-architectural compatibility. On the MASK benchmark, it significantly improves honesty in six out of seven mainstream LLMs. Ablation studies confirm that Gaussian scheduling uniquely disentangles honesty from knowledge retention, outperforming alternative scheduling strategies. This work advances controllable honesty in LLMs through interpretable, parameter-free intervention.

0 citationsRead paper

Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models

Oct 31, 2025

Open-weight foundation models in biology pose dual-use risks: while accelerating scientific discovery and drug development, they may also be misused for bioweapon design. Current mitigation strategies rely primarily on filtering biohazardous data during pretraining, yet their robustness against malicious fine-tuning remains unassessed. This work introduces the first systematic evaluation framework targeting three critical capabilities—viral sequence modeling, mutational effect prediction, and virulence forecasting—employing fine-grained task benchmarks, adversarial fine-tuning experiments, and linear probe analysis. Results demonstrate that: (1) filtered hazardous knowledge is readily recoverable via fine-tuning; (2) dual-use signals are linearly decodable from model representations; and (3) existing data filtering fails to prevent targeted misuse. These findings expose fundamental limitations of current safety mechanisms and provide empirical evidence and methodological foundations for developing more robust governance strategies for open biological foundation models.

0 citationsRead paper