🤖 AI Summary
This paper systematically analyzes four core safety risks of large language models (LLMs): factual inconsistency, intrinsic bias, lack of traceability, and adversarial robustness—manifesting as hallucination, embedded societal biases, undetectable AI-generated content, and prompt injection/jailbreaking attacks. Methodologically, it introduces the first integrated evaluation framework combining red-teaming, controllable input assessment, and a full-lifecycle bias governance pipeline spanning pretraining, fine-tuning, and deployment. It critically benchmarks DetectGPT, watermarking, and ML-based detectors by analyzing their applicability boundaries using HackAPrompt competition data and multi-source factual verification. The primary contribution is the construction of the first holistic LLM safety evaluation landscape jointly optimizing accuracy, fairness, traceability, and robustness; it identifies critical research gaps in cross-stage coordinated defense and proposes actionable technical pathways and governance recommendations.
📝 Abstract
Large Language Models (LLMs) demonstrate impressive capabilities across various fields, yet their increasing use raises critical security concerns. This article reviews recent literature addressing key issues in LLM security, with a focus on accuracy, bias, content detection, and vulnerability to attacks. Issues related to inaccurate or misleading outputs from LLMs is discussed, with emphasis on the implementation from fact-checking methodologies to enhance response reliability. Inherent biases within LLMs are critically examined through diverse evaluation techniques, including controlled input studies and red teaming exercises. A comprehensive analysis of bias mitigation strategies is presented, including approaches from pre-processing interventions to in-training adjustments and post-processing refinements. The article also probes the complexity of distinguishing LLM-generated content from human-produced text, introducing detection mechanisms like DetectGPT and watermarking techniques while noting the limitations of machine learning enabled classifiers under intricate circumstances. Moreover, LLM vulnerabilities, including jailbreak attacks and prompt injection exploits, are analyzed by looking into different case studies and large-scale competitions like HackAPrompt. This review is concluded by retrospecting defense mechanisms to safeguard LLMs, accentuating the need for more extensive research into the LLM security field.