BioSentinel at EXIST 2026: Soft-Label Optimization with XLM-RoBERTa for Sexism Intent Classification in Memes

2026-07-27

AI Summary

This work addresses the detection of gender bias in online memes by classifying instances into direct, judgmental, or non-gender-discriminatory categories, while simultaneously predicting both hard labels and soft labels in the form of probability distributions. Building upon the XLM-RoBERTa architecture, the proposed approach employs a composite loss function that integrates KL divergence with weighted cross-entropy within the Learning with Disagreement framework to explicitly model annotator disagreement. Temperature scaling is further incorporated to refine soft-label predictions. This methodology enhances modeling capacity for subjective tasks. On the official test set, the model achieves an ICM-Soft-Norm score of 0.3229, an ICM-Norm score of 0.3778, and a hard-label F1-score of 0.4236, ranking 40th out of 118 teams in soft–soft evaluation and 49th out of 187 in hard–hard evaluation.

Abstract

This paper describes the BioSentinel team's participation in EXIST 2026 Task 2.2: Source Intention in Memes, part of the CLEF 2026 evaluation campaign. The task requires classifying the communicative intent behind memes as direct, judgemental, or no (non-sexist), under a Learning with Disagreement (Le-Wi-Di) paradigm that mandates both hard-label and soft-label (probability distribution) predictions. We present a text-centric approach built on xlm-roberta-base (270M parameters) trained with a composite loss function combining KL divergence on soft annotator distributions and weighted cross-entropy on hard labels. On the official test set, the system achieved an ICM-Soft-Norm of 0.3229 and ICM-Norm of 0.3778, with a hard F1-score of 0.4236, ranking 40th (out of 118 submissions) in the soft-soft evaluation and 49th (out of 187 submissions) in the hard-hard evaluation. We provide an analysis of the dataset characteristics, exploratory larger-architecture runs, and the role of annotator disagreement in shaping model design for subjective NLP tasks. Ablation results show that KL loss improves soft-label metrics, while CE loss improves hard-label accuracy. We also report a separate validation-set temperature analysis.

Problem

• sexism intent classification
• memes
• soft-label prediction
• annotator disagreement
• communicative intent

Innovation

• soft-label optimization
• XLM-RoBERTa
• Learning with Disagreement
• KL divergence
• meme sexism classification