🤖 AI Summary
This work investigates whether large language models (LLMs) possess intrinsic moral self-correction capabilities. We propose the “Self-Discrimination” validation framework, integrating hidden-state intervention analysis, natural-language weak-evidence perturbation, multi-component ablation studies, and task-based evaluation to systematically dissect the interplay among chain-of-thought reasoning, external feedback, and instruction prompting in moral representation correction. Our mechanistic analysis establishes, for the first time: (1) moral self-correction is not an inherent property of pretraining; (2) internal knowledge and external feedback exhibit negative interaction; (3) while models can revise erroneous outputs, they lack reliable discrimination between high- and low-quality responses; and (4) no universally optimal correction strategy exists. The core contributions lie in empirically demonstrating the non-innateness, mechanism-dependence, and evaluation fragility of moral correction—providing novel empirical foundations for trustworthy AI alignment.
📝 Abstract
Though intensive attentions to the self-correction capability of Large Language Models (LLMs), the underlying mechanism of this capability is still under-explored. In this paper, we aim to answer two fundamental questions for moral self-correction: (1) how different components in self-correction, such as Chain-of-Thought (CoT) reasoning, external feedback, and instructional prompts, interact to enable moral self-correction; and (2) is the self-correction one of LLMs' innate capabilities? To answer the first question, we examine how different self-correction components interact to intervene the embedded morality within hidden states, therefore contributing to different performance. For the second question, we (i) evaluate the robustness of moral self-correction by introducing natural language interventions of weak evidence into prompts; (ii) propose a validation framework, self-distinguish, that requires effective self-correction to enable LLMs to distinguish between desirable and undesirable outputs. Our experimental results indicate that there is no universally optimal self-correction method for the tasks considered, although external feedback and CoT can contribute to additional performance gains. However, our mechanistic analysis reveals negative interactions among instructional prompts, CoT, and external feedback, suggesting a conflict between internal knowledge and external feedback. The self-distinguish experiments demonstrate that while LLMs can self-correct their responses, they are unable to reliably distinguish between desired and undesired outputs. With our empirical evidence, we can conclude that moral self-correction is not an innate capability of LLMs acquired during pretraining.