CHASM: Unveiling Covert Advertisements on Chinese Social Media

2026-04-22

AI Summary

This study addresses the challenge that current large language models struggle to effectively detect covert advertisements on social media disguised as ordinary user content, posing ethical and legal risks. The work presents the first systematic formulation of the covert ad detection task for Chinese social platforms and introduces CHASM, a high-quality multimodal benchmark dataset comprising 4,992 real, anonymized, and human-annotated samples from Xiaohongshu, with strict privacy safeguards. Using this dataset, the authors evaluate the performance of prominent multimodal large language models under zero-shot, in-context learning, and fine-tuning settings. Experimental results reveal that existing models perform poorly in zero-shot and in-context learning scenarios; while fine-tuning substantially improves performance, models still fail to adequately capture subtle cues in comments and structural discrepancies between text and images. This work establishes the first standardized benchmark and empirical analysis for covert advertisement detection in Chinese social media.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Current benchmarks for evaluating large language models (LLMs) in social media moderation completely overlook a serious threat: covert advertisements, which disguise themselves as regular posts to deceive and mislead consumers into making purchases, leading to significant ethical and legal concerns. In this paper, we present the CHASM, a first-of-its-kind dataset designed to evaluate the capability of Multimodal Large Language Models (MLLMs) in detecting covert advertisements on social media. CHASM is a high-quality, anonymized, manually curated dataset consisting of 4,992 instances, based on real-world scenarios from the Chinese social media platform Rednote. The dataset was collected and annotated under strict privacy protection and quality control protocols. It includes many product experience sharing posts that closely resemble covert advertisements, making the dataset particularly challenging.The results show that under both zero-shot and in-context learning settings, none of the current MLLMs are sufficiently reliable for detecting covert advertisements.Our further experiments revealed that fine-tuning open-source MLLMs on our dataset yielded noticeable performance gains. However, significant challenges persist, such as detecting subtle cues in comments and differences in visual and textual structures.We provide in-depth error analysis and outline future research directions. We hope our study can serve as a call for the research community and platform moderators to develop more precise defenses against this emerging threat.

Problem

• covert advertisements
• social media moderation
• multimodal large language models
• ethical concerns
• deceptive content

Innovation

• covert advertisements
• multimodal large language models
• social media moderation
• CHASM dataset
• fine-tuning