Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts

2026-04-11

AI Summary

This study addresses the critical gap in existing evaluation methods that overlook the visual modality, thereby failing to comprehensively detect social biases of vision-language models in educational contexts. Drawing on the tripartite model of attitudes from social psychology, this work proposes the first multimodal bias auditing framework structured around cognitive, affective, and behavioral dimensions. It integrates a self-correcting generation pipeline with a human-in-the-loop validation mechanism to synthesize contamination-resistant student personas for stress-testing mainstream models. The findings reveal that visual inputs can circumvent textual alignment safeguards, triggering subtle and counterintuitive bias resurgences—such as compensatory socioeconomic preferences and deep-seated health- or race-based stereotypes—demonstrating that the visual channel acts as a “safe backdoor” for bias and exposing a systemic misalignment between model cognition and decision-making.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

As Vision-Language Models (VLMs) become integral to educational decision-making, ensuring their fairness is paramount. However, current text-centric evaluations neglect the visual modality, leaving an unregulated channel for latent social biases. To bridge this gap, we present Edu-MMBias, a systematic auditing framework grounded in the tri-component model of attitudes from social psychology. This framework diagnoses bias across three hierarchical dimensions: cognitive, affective, and behavioral. Utilizing a specialized generative pipeline that incorporates a self-correct mechanism and human-in-the-loop verification, we synthesize contamination-resistant student profiles to conduct a holistic stress test on state-of-the-art VLMs. Our extensive audit reveals critical, counter-intuitive patterns: models exhibit a compensatory class bias favoring lower-status narratives while simultaneously harboring deep-seated health and racial stereotypes. Crucially, we find that visual inputs act as a safety backdoor, triggering a resurgence of biases that bypass text-based alignment safeguards and revealing a systematic misalignment between latent cognition and final decision-making. The contributions of this paper are available at: https://anonymous.4open.science/r/EduMMBias-63B2.

Problem

• social bias
• vision-language models
• multimodal benchmark
• educational contexts
• fairness auditing

Innovation

• multimodal bias auditing
• vision-language models
• educational fairness
• visual backdoor
• self-corrective generation