RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generation

2026-04-19

AI Summary

This work addresses the challenge of detecting harmful content in multi-turn dialogues, which requires full contextual understanding yet suffers from inconsistent and poorly interpretable judgments when relying solely on a model’s internal knowledge without explicit moral references. To overcome this limitation, the authors propose RoTRAG, a novel framework that integrates human-authored, concise moral guidelines—termed “Rules of Thumb”—as external evidence via a retrieval-augmented generation (RAG) mechanism. This enables large language models to perform turn-by-turn reasoning and final harm classification grounded in explicit ethical norms. RoTRAG further incorporates a lightweight routing classifier that dynamically determines whether to activate retrieval, balancing computational efficiency with performance. Evaluated on the ProsocialDialog and Safety Reasoning Multi-Turn Dialogue datasets, the approach achieves an average relative F1 improvement of approximately 40%, reduces distributional error by 8.4%, and minimizes redundant computation.

Technology Category

• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Detecting harmful content in multi turn dialogue requires reasoning over the full conversational context rather than isolated utterances. However, most existing methods rely mainly on models internal parametric knowledge, without explicit grounding in external normative principles. This often leads to inconsistent judgments in socially nuanced contexts, limited interpretability, and redundant reasoning across turns. To address this, we propose RoTRAG, a retrieval augmented framework that incorporates concise human written moral norms, called Rules of Thumb (RoTs), into LLM based harm assessment. For each turn, RoTRAG retrieves relevant RoTs from an external corpus and uses them as explicit normative evidence for turn level reasoning and final severity classification. To improve efficiency, we further introduce a lightweight binary routing classifier that decides whether a new turn requires retrieval grounded reasoning or can reuse existing context. Experiments on ProsocialDialog and Safety Reasoning Multi Turn Dialogue show that RoTRAG consistently improves both harm classification and severity estimation over competitive baselines, with an average relative gain of around 40% in F1 across benchmark datasets and an average relative reduction of 8.4% in distributional error, while reducing redundant computation without sacrificing performance.

Problem

• harm detection
• multi-turn dialogue
• normative reasoning
• retrieval-augmented generation
• interpretability

Innovation

• Retrieval-Augmented Generation
• Rule of Thumb
• Harm Detection
• Multi-turn Dialogue
• Normative Reasoning