🤖 AI Summary
Current red-teaming approaches for large language models (LLMs) rely heavily on manual efforts or static datasets, resulting in low efficiency and limited capacity to uncover deep-seated security vulnerabilities. This work proposes the first automatic and adaptive red-teaming framework based on Generative Flow Networks (GFlowNets), which leverages an attacker LLM to dynamically generate highly creative adversarial inputs. The framework autonomously identifies vulnerabilities in target models and quantifies their robustness without human intervention. By introducing GFlowNets into LLM red-teaming for the first time, the method outperforms existing benchmarks in English attack generation and pioneers support for automatic adversarial input generation in low-resource languages such as Turkish, substantially enhancing test coverage and evaluation efficiency.
📝 Abstract
The rapid advancement of Large Language Models (LLMs) has facilitated their ubiquitous integration into various domains, leading to widespread adoption. However, this escalating trend has introduced significant security vulnerabilities, necessitating the identification and mitigation of flaws arising from malicious exploitation. Red teaming assessments, conducted to evaluate model robustness through diverse adversarial inputs, are essential for exposing security risks and implementing countermeasures. Currently, red teaming is performed either manually by experts or automatically using predefined attack datasets. Nevertheless, manual testing remains time-consuming, while existing automated methods suffer from limited creativity due to their inherent dependency on fixed datasets. In this study, we propose an automated, human-independent, and adaptive approach leveraging GFlowNets to identify LLM vulnerabilities by utilizing one large language model to test another. Within this framework, an attacker model is trained against a specified victim model to perform automated red teaming and provide a quantitative robustness score. This research aims to generate more effective adversarial attacks in English compared to existing benchmarks and, as a novel contribution to the literature, introduces a model capable of generating attack inputs in the Turkish language.