System Report for CCL25-Eval Task 10: SRAG-MAV for Fine-Grained Chinese Hate Speech Recognition

2025-07-24

AI Summary

This work addresses fine-grained Chinese hate speech recognition (FGCHSR). To tackle its complexity, we propose SRAG-MAV: (1) reformulating the conventional quadruple extraction task into a simplified triplet-based structure; (2) introducing a self-retrieval-augmented generation (SRAG) mechanism that dynamically retrieves semantically relevant in-context examples to construct adaptive prompts; and (3) incorporating a multi-round accumulated voting (MAV) strategy to enhance reasoning consistency and output robustness. Implemented on Qwen2.5-7B, SRAG-MAV achieves an average score of 37.505 on ToxiCN (hard score: 26.66; soft score: 48.35), substantially outperforming GPT-4o (15.63) and fine-tuned Qwen2.5-7B (35.365). These results empirically validate the effectiveness of jointly optimizing task restructuring, dynamic retrieval, and iterative consensus voting for FGCHSR.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Adversarial Learning & Robustness
• Knowledge Representation and Reasoning: Argumentation

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

This paper presents our system for CCL25-Eval Task 10, addressing Fine-Grained Chinese Hate Speech Recognition (FGCHSR). We propose a novel SRAG-MAV framework that synergistically integrates task reformulation(TR), Self-Retrieval-Augmented Generation (SRAG), and Multi-Round Accumulative Voting (MAV). Our method reformulates the quadruplet extraction task into triplet extraction, uses dynamic retrieval from the training set to create contextual prompts, and applies multi-round inference with voting to improve output stability and performance. Our system, based on the Qwen2.5-7B model, achieves a Hard Score of 26.66, a Soft Score of 48.35, and an Average Score of 37.505 on the STATE ToxiCN dataset, significantly outperforming baselines such as GPT-4o (Average Score 15.63) and fine-tuned Qwen2.5-7B (Average Score 35.365). The code is available at https://github.com/king-wang123/CCL25-SRAG-MAV.

Problem

• Fine-grained Chinese hate speech recognition challenge
• Quadruplet to triplet extraction task reformulation
• Improving output stability with multi-round voting

Innovation

• Reformulates quadruplet extraction into triplet extraction
• Uses dynamic retrieval for contextual prompts
• Applies multi-round voting for stable outputs