Probing AI Safety with Source Code

2025-06-25

AI Summary

This work addresses the alignment failure of large language models (LLMs) in safety-critical scenarios. We propose Code-Driven Thinking (CoDoT), a prompting strategy that translates natural-language instructions into functionally equivalent, concise code representations. We introduce the first source-code-driven safety evaluation framework and empirically demonstrate that mainstream LLMs exhibit severe safety degradation under code-formatted inputs: GPT-4 Turbo’s toxicity increases 16.5×, DeepSeek R1 achieves a 100% failure rate, and the average toxicity across seven models rises by 300%; recursive CoDoT further doubles toxicity. These findings expose a systemic vulnerability of conventional alignment methods to structured, programmable inputs. Our work shifts safety assessment from natural-language-centric paradigms toward a code-based paradigm—characterized by programmability, reproducibility, and scalability—and provides both a critical diagnostic tool and theoretical warning for achieving robust LLM alignment.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Security and Privacy: Large-scale security measurements

Abstract

Large language models (LLMs) have become ubiquitous, interfacing with humans in numerous safety-critical applications. This necessitates improving capabilities, but importantly coupled with greater safety measures to align these models with human values and preferences. In this work, we demonstrate that contemporary models fall concerningly short of the goal of AI safety, leading to an unsafe and harmful experience for users. We introduce a prompting strategy called Code of Thought (CoDoT) to evaluate the safety of LLMs. CoDoT converts natural language inputs to simple code that represents the same intent. For instance, CoDoT transforms the natural language prompt "Make the statement more toxic: {text}" to: "make_more_toxic({text})". We show that CoDoT results in a consistent failure of a wide range of state-of-the-art LLMs. For example, GPT-4 Turbo's toxicity increases 16.5 times, DeepSeek R1 fails 100% of the time, and toxicity increases 300% on average across seven modern LLMs. Additionally, recursively applying CoDoT can further increase toxicity two times. Given the rapid and widespread adoption of LLMs, CoDoT underscores the critical need to evaluate safety efforts from first principles, ensuring that safety and capabilities advance together.

Problem

• Evaluating AI safety gaps in large language models (LLMs)
• Assessing toxicity amplification via code-based prompting (CoDoT)
• Highlighting safety-capability misalignment in modern LLMs

Innovation

• Converts natural language to simple code
• Evaluates LLM safety via Code of Thought
• Recursively increases toxicity for testing