RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experiments

2026-06-24

AI Summary

This work investigates how to reverse-engineer executable decision code of an agent solely from its behavioral trajectories in games and examines how actively designed adversarial experiments can enhance reconstruction fidelity. To this end, we introduce RevengeBench, a benchmark comprising 75 Elo-calibrated CodeClash strategies across five environments, where customized behavioral probes are generated by observing interactions between target and opponent agents to reconstruct underlying code logic. We formalize strategy reversal as a tractable inverse problem in code space and incorporate a mechanism for controlled experimentation. Experiments across 12 large language models demonstrate that our approach significantly reduces initial behavioral divergence (by 34%–72%) and yields reconstructed strategies that exhibit competitive performance in downstream adversarial settings, particularly bolstering the counterplay capabilities of weaker models.

Abstract

For most of scientific history, researchers studying behavior could only infer hidden mechanisms from outward actions: an inverse problem that becomes more tractable when observation is augmented by targeted intervention. We pose a computational analogue: given only behavioral traces of an agent in a game environment, can a learner reconstruct the underlying decision program as executable code, and how much does this reconstruction improve with the ability to design controlled experiments? We introduce RevengeBench, a benchmark of 75 LLM generated, Elo-calibrated policies across five game environments, drawn from CodeClash tournament trajectories. The learner observes the hidden target policy play against sampled opponents and designs behavioral probes in the form of custom opponent policies that elicit informative behavior. It then submits an executable hypothesis, which is evaluated using continuous action-distance metrics. We further validate that recovered code carries informative signal in downstream player-versus-player tournaments. Across twelve frontier LLMs, recovery quality varies substantially (34 to 72% of initial distance closed), with reconstructed policies yielding measurable competitive advantage, particularly for weaker models that otherwise struggle to design effective counter-strategies. Our benchmark positions behavioral recovery of programmatic policies as a tractable inverse problem in code-space, opening a path to opponent modeling, policy interpretability, and the broader question of inferring latent mechanisms from observations.

Problem

• reverse engineering
• behavioral inference
• programmatic policies
• inverse problem
• opponent modeling

Innovation

• reverse engineering
• programmatic policies
• behavioral probing
• code-space inference
• opponent modeling