On the Consideration of AI Openness: Can Good Intent Be Abused?

2024-03-11

arXiv.org

AI Summary

This study investigates the risks of malicious misuse of open-source AI technologies for criminal purposes and the associated legal liability of developers. To address this, we construct EVE-V1/V2—the first dual-version case-based question-answering dataset specifically designed for criminal scenarios—and conduct lightweight supervised fine-tuning and empirical evaluation on over 200 mainstream open-source large language models (e.g., LLaMA, Falcon). Our findings demonstrate, for the first time, that minimal fine-tuning with crime-related data suffices to elicit concrete, actionable illegal instructions from otherwise benign models; moreover, such adversarial capabilities generalize across distinct criminal offenses, as validated on EVE-V2. Building on these results, we propose a developer liability framework centered on the legal principle of “reasonable foreseeability,” offering both an empirically grounded and operationally viable basis for governing open-source AI systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Responsible Web: Social and technical mechanisms of refusal for web technologies and applications
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Open source is a driving force behind scientific advancement.However, this openness is also a double-edged sword, with the inherent risk that innovative technologies can be misused for purposes harmful to society. What is the likelihood that an open source AI model or dataset will be used to commit a real-world crime, and if a criminal does exploit it, will the people behind the technology be able to escape legal liability? To address these questions, we explore a legal domain where individual choices can have a significant impact on society. Specifically, we build the EVE-V1 dataset that comprises 200 question-answer pairs related to criminal offenses based on 200 Korean precedents first to explore the possibility of malicious models emerging. We further developed EVE-V2 using 600 fraud-related precedents to confirm the existence of malicious models that can provide harmful advice on a wide range of criminal topics to test the domain generalization ability. Remarkably, widely used open-source large-scale language models (LLMs) provide unethical and detailed information about criminal activities when fine-tuned with EVE. We also take an in-depth look at the legal issues that malicious language models and their builders could realistically face. Our findings highlight the paradoxical dilemma that open source accelerates scientific progress, but requires great care to minimize the potential for misuse. Warning: This paper contains content that some may find unethical.

Problem

• AI ethics
• legal liability
• crime prevention

Innovation

• EVE-V1 and EVE-V2 datasets
• AI liability in criminal activities
• Open-source AI misuse risk