Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLM

2025-08-19

AI Summary

This study introduces speaker assignment as a novel lens for detecting gender bias in speech large language models (Speech-LLMs). Focusing on the text-to-speech model Bark, we construct a benchmark dataset comprising gender-stereotyped occupations and gendered lexical items, and systematically analyze the gender distribution of default synthesized speech—i.e., without explicit speaker prompts. Through rigorous empirical statistical testing, we find that while Bark exhibits no statistically significant systematic gender bias, it consistently demonstrates gender awareness and measurable gendered preferences, suggesting latent gender associations embedded in its training data or architecture. Crucially, this work pioneers the conceptualization of speaker assignment as a bias detection interface, establishing a reproducible and scalable methodological framework for fairness evaluation of Speech-LLMs.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Ethics, Bias, and Fairness
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Similar to text-based Large Language Models (LLMs), Speech-LLMs exhibit emergent abilities and context awareness. However, whether these similarities extend to gender bias remains an open question. This study proposes a methodology leveraging speaker assignment as an analytic tool for bias investigation. Unlike text-based models, which encode gendered associations implicitly, Speech-LLMs must produce a gendered voice, making speaker selection an explicit bias cue. We evaluate Bark, a Text-to-Speech (TTS) model, analyzing its default speaker assignments for textual prompts. If Bark's speaker selection systematically aligns with gendered associations, it may reveal patterns in its training data or model design. To test this, we construct two datasets: (i) Professions, containing gender-stereotyped occupations, and (ii) Gender-Colored Words, featuring gendered connotations. While Bark does not exhibit systematic bias, it demonstrates gender awareness and has some gender inclinations.

Problem

• Investigating gender bias in Speech-LLM speaker assignment
• Evaluating gendered voice selection in Text-to-Speech models
• Analyzing systematic alignment with gender stereotypes in outputs

Innovation

• Speaker assignment as bias analytic tool
• Evaluating Bark TTS model default assignments
• Constructed profession and gender-colored datasets