FinSafetyBench: Evaluating LLM Safety in Real-World Financial Scenarios

2026-05-01

AI Summary

This study addresses the critical compliance risks posed by large language models (LLMs) in financial contexts, where model outputs may inadvertently facilitate illicit activities or violate ethical norms. To systematically evaluate the safety of both general-purpose and finance-specific LLMs, the authors construct the first bilingual (Chinese–English) red-teaming benchmark grounded in real-world financial crime cases and established ethical guidelines, encompassing 14 categories of regulatory violations. Through three adversarial attack settings, the research assesses models’ ability to refuse harmful requests. Findings reveal significant security vulnerabilities across current models, particularly in Chinese-language interactions, where implicit adversarial prompts frequently bypass compliance safeguards—highlighting fundamental limitations in existing prompt-level defense mechanisms.

Abstract

Large language models (LLMs) are increasingly applied in financial scenarios. However, they may produce harmful outputs, including facilitating illegal activities or unethical behavior, posing serious compliance risks. To systematically evaluate LLM safety in finance, we propose FinSafetyBench, a bilingual (English-Chinese) red-teaming benchmark designed to test an LLM's refusal of requests that violate financial compliance. Grounded in real-world financial crime cases and ethics standards, the benchmark comprises 14 subcategories spanning financial crimes and ethical violations. Through extensive experiments on general-purpose and finance-specialized LLMs under three representative attack settings, we identify critical vulnerabilities that allow adversarial prompts to bypass compliance safeguards. Further analysis reveals stronger susceptibility in Chinese contexts and highlights the limitations of prompt-level defenses against sophisticated or implicit manipulation strategies.

Problem

• LLM safety
• financial compliance
• red-teaming
• adversarial prompts
• ethical violations

Innovation

• FinSafetyBench
• red-teaming benchmark
• financial compliance
• bilingual evaluation
• adversarial prompts