Mario at EXIST 2025: A Simple Gateway to Effective Multilingual Sexism Detection

2025-07-15

AI Summary

This work addresses gender bias detection in English–Spanish bilingual tweets through a hierarchical multi-task framework that jointly performs binary classification, intent source localization, and fine-grained multi-label classification. To explicitly model label dependencies across the three task levels, we propose a novel conditional adapter routing mechanism. Leveraging Llama-3.1-8B, we apply hierarchical LoRA (rank=16) with QLoRA 4-bit quantization to all linear layers, enabling lightweight multilingual joint fine-tuning. Our approach introduces only 1.67% trainable parameters, reduces training time by 75%, and compresses model size by 98%. On the three subtasks, it achieves F1 scores of 0.6774, 0.4991, and 0.6519—outperforming monolingual baselines by 1.7–2.4 percentage points—demonstrating the efficacy of cross-lingual knowledge transfer and synergistic task modeling.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This paper presents our approach to EXIST 2025 Task 1, addressing text-based sexism detection in English and Spanish tweets through hierarchical Low-Rank Adaptation (LoRA) of Llama 3.1 8B. Our method introduces conditional adapter routing that explicitly models label dependencies across three hierarchically structured subtasks: binary sexism identification, source intention detection, and multilabel sexism categorization. Unlike conventional LoRA applications that target only attention layers, we apply adaptation to all linear transformations, enhancing the model's capacity to capture task-specific patterns. In contrast to complex data processing and ensemble approaches, we show that straightforward parameter-efficient fine-tuning achieves strong performance. We train separate LoRA adapters (rank=16, QLoRA 4-bit) for each subtask using unified multilingual training that leverages Llama 3.1's native bilingual capabilities. The method requires minimal preprocessing and uses standard supervised learning. Our multilingual training strategy eliminates the need for separate language-specific models, achieving 1.7-2.4% F1 improvements through cross-lingual transfer. With only 1.67% trainable parameters compared to full fine-tuning, our approach reduces training time by 75% and model storage by 98%, while achieving competitive performance across all subtasks (ICM-Hard: 0.6774 for binary classification, 0.4991 for intention detection, 0.6519 for multilabel categorization).

Problem

• Detect text-based sexism in multilingual tweets
• Model hierarchical subtasks for sexism classification
• Improve efficiency with parameter-efficient fine-tuning

Innovation

• Hierarchical LoRA adapters for multilingual sexism detection
• Conditional adapter routing models label dependencies
• Efficient 4-bit QLoRA fine-tuning with minimal preprocessing