FHSTP@EXIST 2025 Benchmark: Sexism Detection with Transparent Speech Concept Bottleneck Models

2025-07-28

AI Summary

This study addresses the fine-grained identification and interpretable classification of gender-discriminatory language in social media text. Methodologically, it proposes the Speech Concept Bottleneck Model (SCBM) and its enhanced variant SCBMT, which innovatively incorporate human-understandable descriptive adjectives—generated by large language models—as bottleneck concepts, fused with Transformer-based contextual embeddings. The framework further integrates multilingual data augmentation and meta-analysis to support cross-lingual detection (English/Spanish). Unlike opaque black-box models, SCBM/SCBMT provide instance-level and class-level explanations that are both transparent and traceable. Experiments show that fine-tuned XLM-RoBERTa ranks fourth on the English subtask, while SCBMT achieves seventh place on the bilingual (English–Spanish) subtask—demonstrating competitive performance alongside substantially improved model transparency and interpretability. This work advances responsible AI governance through a novel, concept-driven, and linguistically grounded approach to bias detection.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Transparent, Interpretable, Explainable ML
• Humans and AI: Explainable AI (XAI) for Human Understanding

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Sexism has become widespread on social media and in online conversation. To help address this issue, the fifth Sexism Identification in Social Networks (EXIST) challenge is initiated at CLEF 2025. Among this year's international benchmarks, we concentrate on solving the first task aiming to identify and classify sexism in social media textual posts. In this paper, we describe our solutions and report results for three subtasks: Subtask 1.1 - Sexism Identification in Tweets, Subtask 1.2 - Source Intention in Tweets, and Subtask 1.3 - Sexism Categorization in Tweets. We implement three models to address each subtask which constitute three individual runs: Speech Concept Bottleneck Model (SCBM), Speech Concept Bottleneck Model with Transformer (SCBMT), and a fine-tuned XLM-RoBERTa transformer model. SCBM uses descriptive adjectives as human-interpretable bottleneck concepts. SCBM leverages large language models (LLMs) to encode input texts into a human-interpretable representation of adjectives, then used to train a lightweight classifier for downstream tasks. SCBMT extends SCBM by fusing adjective-based representation with contextual embeddings from transformers to balance interpretability and classification performance. Beyond competitive results, these two models offer fine-grained explanations at both instance (local) and class (global) levels. We also investigate how additional metadata, e.g., annotators' demographic profiles, can be leveraged. For Subtask 1.1, XLM-RoBERTa, fine-tuned on provided data augmented with prior datasets, ranks 6th for English and Spanish and 4th for English in the Soft-Soft evaluation. Our SCBMT achieves 7th for English and Spanish and 6th for Spanish.

Problem

• Identify and classify sexism in social media posts
• Develop interpretable models for sexism detection
• Leverage metadata to enhance classification performance

Innovation

• Speech Concept Bottleneck Model for interpretability
• Transformer fusion for balanced performance
• Fine-tuned XLM-RoBERTa for multilingual classification