Certainly Bot Or Not? Trustworthy Social Bot Detection via Robust Multi-Modal Neural Processes

2025-03-11

AI Summary

Existing social bot detectors suffer from overconfident predictions under cross-dataset distribution shifts and struggle to resolve multimodal information conflicts induced by adversarial camouflage, resulting in poor robustness and reliability. To address these challenges, we propose an uncertainty-aware robust multimodal neural process framework. Our method introduces a novel modality reliability modeling mechanism that integrates evidential gating with generalized Product-of-Experts (PoE), coupled with modality-specific encoders, unimodal attention-based neural processes, and Monte Carlo sampling over latent variables for decoding—jointly optimizing uncertainty quantification and camouflage resistance. Evaluated on three real-world benchmarks, our approach significantly improves classification accuracy and prediction calibration, demonstrating strong generalization and robustness against out-of-distribution samples and multimodal conflict scenarios.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Safety and Robustness
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Social bot detection is crucial for mitigating misinformation, online manipulation, and coordinated inauthentic behavior. While existing neural network-based detectors perform well on benchmarks, they struggle with generalization due to distribution shifts across datasets and frequently produce overconfident predictions for out-of-distribution accounts beyond the training data. To address this, we introduce a novel Uncertainty Estimation for Social Bot Detection (UESBD) framework, which quantifies the predictive uncertainty of detectors beyond mere classification. For this task, we propose Robust Multi-modal Neural Processes (RMNP), which aims to enhance the robustness of multi-modal neural processes to modality inconsistencies caused by social bot camouflage. RMNP first learns unimodal representations through modality-specific encoders. Then, unimodal attentive neural processes are employed to encode the Gaussian distribution of unimodal latent variables. Furthermore, to avoid social bots stealing human features to camouflage themselves thus causing certain modalities to provide conflictive information, we introduce an evidential gating network to explicitly model the reliability of modalities. The joint latent distribution is learned through the generalized product of experts, which takes the reliability of each modality into consideration during fusion. The final prediction is obtained through Monte Carlo sampling of the joint latent distribution followed by a decoder. Experiments on three real-world benchmarks show the effectiveness of RMNP in classification and uncertainty estimation, as well as its robustness to modality conflicts.

Problem

• Detecting social bots to combat misinformation and manipulation.
• Improving generalization of bot detection across datasets.
• Enhancing robustness against modality inconsistencies in bot detection.

Innovation

• Uncertainty Estimation for Social Bot Detection
• Robust Multi-modal Neural Processes
• Evidential gating network for modality reliability