VEAT Quantifies Implicit Associations in Text-to-Video Generator Sora and Reveals Challenges in Bias Mitigation

2026-01-02

arXiv.org

AI Summary

This study addresses the lack of effective methods for evaluating implicit social biases in current text-to-video generation models such as Sora. It proposes the Video Embedding Association Test (VEAT) and its single-category variant (SC-VEAT), extending the Implicit Association Test to the video domain by integrating video semantic analysis, effect size estimation, and correlation metrics to systematically quantify racial and gender biases in Sora. The findings reveal that Sora strongly associates White Americans and women with positive emotions (Cohen’s d > 0.8), and the magnitude of these biases closely correlates with real-world demographic disparities in occupations and awards (r = 0.83–0.99). Moreover, while existing explicit debiasing prompts can mitigate certain biases, they may inadvertently exacerbate stereotypical representations of Black individuals in specific occupational contexts.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Text-to-Video (T2V) generators such as Sora raise concerns about whether generated content reflects societal bias. We extend embedding-association tests from words and images to video by introducing the Video Embedding Association Test (VEAT) and Single-Category VEAT (SC-VEAT). We validate these methods by reproducing the direction and magnitude of associations from widely used baselines, including Implicit Association Test (IAT) scenarios and OASIS image categories. We then quantify race (African American vs. European American) and gender (women vs. men) associations with valence (pleasant vs. unpleasant) across 17 occupations and 7 awards. Sora videos associate European Americans and women more with pleasantness (both d>0.8). Effect sizes correlate with real-world demographic distributions: percent men and White in occupations (r=0.93, r=0.83) and percent male and non-Black among award recipients (r=0.88, r=0.99). Applying explicit debiasing prompts generally reduces effect-size magnitudes, but can backfire: two Black-associated occupations (janitor, postal service) become more Black-associated after debiasing. Together, these results reveal that easily accessible T2V generators can actually amplify representational harms if not rigorously evaluated and responsibly deployed.

Problem

• bias
• text-to-video generation
• implicit associations
• representation harm
• debiasing

Innovation

• Video Embedding Association Test
• Text-to-Video Generation
• Bias Quantification
• Debiasing Prompts
• Representational Harm