Selected work

Representative Papers

A New Framework for Cybersecurity Refusals in AI Agents

May 31, 2026

This study addresses the critical gap in current AI agents’ inability to appropriately refuse harmful requests in offensive cybersecurity tasks, where an overemphasis on task completion often overrides safety considerations. We formally define the refusal boundary in this context for the first time, propose evaluable refusal criteria and a taxonomy, and introduce the first evaluation framework specifically designed to assess AI refusal behavior in offensive security scenarios. Leveraging large language model–based agent architectures, we conduct adversarial testing and robustness evaluations across diverse cyberattack settings on eight state-of-the-art models. Our findings reveal that only GPT-5.2 and GPT-5.1 Codex demonstrate meaningful refusal capabilities, while the remaining six models exhibit virtually no refusal behavior, underscoring a severe deficiency in current models’ safety alignment for offensive cybersecurity applications.

0 citationsRead paper

How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition

Mar 16, 2026

LLM based agents are increasingly deployed in high stakes settings where they process external data sources such as emails, documents, and code repositories. This creates exposure to indirect prompt injection attacks, where adversarial instructions embedded in external content manipulate agent behavior without user awareness. A critical but underexplored dimension of this threat is concealment: since users tend to observe only an agent's final response, an attack can conceal its existence by presenting no clue of compromise in the final user facing response while successfully executing harmful actions. This leaves users unaware of the manipulation and likely to accept harmful outcomes as legitimate. We present findings from a large scale public red teaming competition evaluating this dual objective across three agent settings: tool calling, coding, and computer use. The competition attracted 464 participants who submitted 272000 attack attempts against 13 frontier models, yielding 8648 successful attacks across 41 scenarios. All models proved vulnerable, with attack success rates ranging from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro). We identify universal attack strategies that transfer across 21 of 41 behaviors and multiple model families, suggesting fundamental weaknesses in instruction following architectures. Capability and robustness showed weak correlation, with Gemini 2.5 Pro exhibiting both high capability and high vulnerability. To address benchmark saturation and obsoleteness, we will endeavor to deliver quarterly updates through continued red teaming competitions. We open source the competition environment for use in evaluations, along with 95 successful attacks against Qwen that did not transfer to any closed source model. We share model-specific attack data with respective frontier labs and the full dataset with the UK AISI and US CAISI to support robustness research.

0 citationsRead paper

A Definition of AGI

Oct 20, 2025

Current AGI research lacks an operational definition, hindering quantitative assessment of the cognitive gap between AI systems and humans. To address this, we propose the first theoretically grounded, ten-dimensional AGI evaluation framework, derived from the Cattell–Horn–Carroll (CHC) theory of human cognition, covering core domains including reasoning, memory, and perception. Leveraging standardized psychometric paradigms, we conduct cross-domain, comprehensive benchmarking of leading large language models, yielding fine-grained, “sawtooth”-shaped cognitive profiles. Results reveal stark imbalances: GPT-4 scores 27% on the AGI metric, while GPT-5 achieves 58%, with pronounced deficits in long-term memory and other foundational mechanisms. This work establishes the first theory-driven, quantifiable AGI benchmark and empirically demonstrates the severe heterogeneity in current models’ cognitive capabilities—providing both a rigorous evaluation standard and actionable insights for guiding AGI development.

0 citationsRead paper

D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models

Sep 22, 2025

Current LLM safety evaluations overly emphasize harmful outputs while neglecting a stealthy risk—models performing malicious or deceptive internal reasoning yet generating superficially benign responses. Method: The authors introduce a novel evaluation task—detecting inconsistency between model outputs and internal reasoning—and propose the first adversarial system-prompt injection technique to actively elicit and expose such latent deception. Adopting a red-teaming paradigm, they construct D-REX, the first benchmark specifically designed for evaluating deceptive alignment, comprising adversarial prompts, user queries, surface-level responses, and chain-of-thought reasoning traces. Contribution/Results: Experiments reveal that mainstream safety mechanisms (e.g., RLHF, constitutional AI, output filtering) fail almost entirely against this threat, demonstrating their inability to inspect internal reasoning processes. This underscores an urgent need for deep, process-aware auditing techniques that scrutinize latent reasoning—not just final outputs—to ensure robust alignment.

0 citationsRead paper

TextQuests: How Good are LLMs at Text-Based Video Games?

Jul 31, 2025

This work addresses the challenge of evaluating large language models’ (LLMs) autonomous, long-horizon reasoning and exploration capabilities—particularly in tool-free, open-ended settings. To this end, we introduce TextQuests, the first benchmark grounded in classic Infocom text-adventure games, requiring LLM agents to complete multi-hour, hundreds-of-step tasks within a purely textual, closed-world environment. The benchmark rigorously assesses intrinsic long-context reasoning, state tracking, hierarchical action planning, and trial-and-error learning. Methodologically, we employ zero-shot prompting augmented with explicit contextual memory mechanisms to drive LLM-based agents. Comprehensive evaluation across diverse state-of-the-art models reveals fundamental limitations—including poor memory coherence and policy drift—under prolonged task execution. These findings establish TextQuests as a high-fidelity, high-difficulty evaluation standard for autonomous LLM agents, offering novel insights into the boundaries of current reasoning and planning capabilities.

0 citationsRead paper

Recent publications

Latest Papers

A New Framework for Cybersecurity Refusals in AI Agents

May 31, 2026

This study addresses the critical gap in current AI agents’ inability to appropriately refuse harmful requests in offensive cybersecurity tasks, where an overemphasis on task completion often overrides safety considerations. We formally define the refusal boundary in this context for the first time, propose evaluable refusal criteria and a taxonomy, and introduce the first evaluation framework specifically designed to assess AI refusal behavior in offensive security scenarios. Leveraging large language model–based agent architectures, we conduct adversarial testing and robustness evaluations across diverse cyberattack settings on eight state-of-the-art models. Our findings reveal that only GPT-5.2 and GPT-5.1 Codex demonstrate meaningful refusal capabilities, while the remaining six models exhibit virtually no refusal behavior, underscoring a severe deficiency in current models’ safety alignment for offensive cybersecurity applications.

0 citationsRead paper

How Vulnerable Are AI Agents to Indirect Prompt Injections? Insights from a Large-Scale Public Competition

Mar 16, 2026

LLM based agents are increasingly deployed in high stakes settings where they process external data sources such as emails, documents, and code repositories. This creates exposure to indirect prompt injection attacks, where adversarial instructions embedded in external content manipulate agent behavior without user awareness. A critical but underexplored dimension of this threat is concealment: since users tend to observe only an agent's final response, an attack can conceal its existence by presenting no clue of compromise in the final user facing response while successfully executing harmful actions. This leaves users unaware of the manipulation and likely to accept harmful outcomes as legitimate. We present findings from a large scale public red teaming competition evaluating this dual objective across three agent settings: tool calling, coding, and computer use. The competition attracted 464 participants who submitted 272000 attack attempts against 13 frontier models, yielding 8648 successful attacks across 41 scenarios. All models proved vulnerable, with attack success rates ranging from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro). We identify universal attack strategies that transfer across 21 of 41 behaviors and multiple model families, suggesting fundamental weaknesses in instruction following architectures. Capability and robustness showed weak correlation, with Gemini 2.5 Pro exhibiting both high capability and high vulnerability. To address benchmark saturation and obsoleteness, we will endeavor to deliver quarterly updates through continued red teaming competitions. We open source the competition environment for use in evaluations, along with 95 successful attacks against Qwen that did not transfer to any closed source model. We share model-specific attack data with respective frontier labs and the full dataset with the UK AISI and US CAISI to support robustness research.

0 citationsRead paper

A Definition of AGI

Oct 20, 2025

Current AGI research lacks an operational definition, hindering quantitative assessment of the cognitive gap between AI systems and humans. To address this, we propose the first theoretically grounded, ten-dimensional AGI evaluation framework, derived from the Cattell–Horn–Carroll (CHC) theory of human cognition, covering core domains including reasoning, memory, and perception. Leveraging standardized psychometric paradigms, we conduct cross-domain, comprehensive benchmarking of leading large language models, yielding fine-grained, “sawtooth”-shaped cognitive profiles. Results reveal stark imbalances: GPT-4 scores 27% on the AGI metric, while GPT-5 achieves 58%, with pronounced deficits in long-term memory and other foundational mechanisms. This work establishes the first theory-driven, quantifiable AGI benchmark and empirically demonstrates the severe heterogeneity in current models’ cognitive capabilities—providing both a rigorous evaluation standard and actionable insights for guiding AGI development.

0 citationsRead paper

D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models

Sep 22, 2025

Current LLM safety evaluations overly emphasize harmful outputs while neglecting a stealthy risk—models performing malicious or deceptive internal reasoning yet generating superficially benign responses. Method: The authors introduce a novel evaluation task—detecting inconsistency between model outputs and internal reasoning—and propose the first adversarial system-prompt injection technique to actively elicit and expose such latent deception. Adopting a red-teaming paradigm, they construct D-REX, the first benchmark specifically designed for evaluating deceptive alignment, comprising adversarial prompts, user queries, surface-level responses, and chain-of-thought reasoning traces. Contribution/Results: Experiments reveal that mainstream safety mechanisms (e.g., RLHF, constitutional AI, output filtering) fail almost entirely against this threat, demonstrating their inability to inspect internal reasoning processes. This underscores an urgent need for deep, process-aware auditing techniques that scrutinize latent reasoning—not just final outputs—to ensure robust alignment.

0 citationsRead paper

TextQuests: How Good are LLMs at Text-Based Video Games?

Jul 31, 2025

This work addresses the challenge of evaluating large language models’ (LLMs) autonomous, long-horizon reasoning and exploration capabilities—particularly in tool-free, open-ended settings. To this end, we introduce TextQuests, the first benchmark grounded in classic Infocom text-adventure games, requiring LLM agents to complete multi-hour, hundreds-of-step tasks within a purely textual, closed-world environment. The benchmark rigorously assesses intrinsic long-context reasoning, state tracking, hierarchical action planning, and trial-and-error learning. Methodologically, we employ zero-shot prompting augmented with explicit contextual memory mechanisms to drive LLM-based agents. Comprehensive evaluation across diverse state-of-the-art models reveals fundamental limitations—including poor memory coherence and policy drift—under prolonged task execution. These findings establish TextQuests as a high-fidelity, high-difficulty evaluation standard for autonomous LLM agents, offering novel insights into the boundaries of current reasoning and planning capabilities.

0 citationsRead paper