Cross-Platform Chinese Offensive Comment Detection via Dual-Threshold Hard Example Mining

2026-06-25

AI Summary

This study addresses the significant performance degradation of offensive comment detection models when deployed across Chinese social media platforms. Building upon RoBERTa, the authors establish a binary classification baseline on the COLD dataset and construct a fine-grained three-class test set spanning Weibo, Xiaohongshu, Tieba, and Zhihu. They propose a dual-threshold hard example mining mechanism that selects high- and low-confidence samples from unlabeled data and combines them with a small set of manually annotated hard examples for a second round of fine-tuning, enabling low-cost domain adaptation. Domain discrepancies are quantified using clean-Chinese-base RoBERTa embeddings alongside Jaccard distance and Proxy-A Distance, and implicit contextual fine-tuning effectively mitigates domain shift. The approach achieves substantial performance gains across all four platforms, overcoming cross-platform transfer bottlenecks with minimal annotation cost.

Abstract

Cross-platform deployment of offensive comment detection for Chinese social media suffers performance degradation. The paper proposes a dual-threshold hard mining method to address this. First, the clean-Chinese-base RoBERTa is finetuned on COLD to establish a binary baseline for fair comparison. Second, a three-class fine-labeled test set covering Weibo, Xiaohongshu, Tieba, and Zhihu is constructed, domain distances from the source are quantified using Jaccard and Proxy-A Distance, as well as the degradation bottleneck of the baseline under domain shift is systematically revealed. Herein, a dual threshold hard example mining strategy is proposed. High- and low-confidence error-prone samples are filtered from unlabeled corpora by prediction confidence. The model is secondarily finetuned under implicit contexts with merely a small set of manually labeled hard examples, realizing low-cost cross-platform domain adaptation. Experiments reveal significant performance gains of the optimized model across four platforms.

Problem

• cross-platform
• offensive comment detection
• domain shift
• Chinese social media
• performance degradation

Innovation

• dual-threshold hard example mining
• cross-platform offensive comment detection
• domain adaptation
• Chinese social media
• RoBERTa fine-tuning