Invisible Filters: Cultural Bias in Hiring Evaluations Using Large Language Models

2025-08-21

AI Summary

This study investigates implicit cultural bias in large language models (LLMs) used for cross-cultural hiring assessments. Method: Using 100 interview transcripts each from the UK and India, we conducted controlled identity-anonymization and attribute-substitution experiments—manipulating gender, caste, and regional origin—and quantified LLM-generated employability scores. Contribution/Results: Indian candidates consistently received significantly lower scores, even when explicit identifiers (e.g., names) were removed. Further analysis revealed that linguistic style features—including syntactic complexity and lexical diversity—exhibited stronger predictive power for score disparities than identity cues, constituting a culture-linked, non-identity-driven bias mechanism. To our knowledge, this is the first systematic identification and empirical validation of language-mediated cultural bias in LLM-based hiring evaluation. The findings call for a novel fairness-by-design paradigm integrating linguistic and sociocultural dimensions to mitigate such biases.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Artificial Intelligence (AI) is increasingly used in hiring, with large language models (LLMs) having the potential to influence or even make hiring decisions. However, this raises pressing concerns about bias, fairness, and trust, particularly across diverse cultural contexts. Despite their growing role, few studies have systematically examined the potential biases in AI-driven hiring evaluation across cultures. In this study, we conduct a systematic analysis of how LLMs assess job interviews across cultural and identity dimensions. Using two datasets of interview transcripts, 100 from UK and 100 from Indian job seekers, we first examine cross-cultural differences in LLM-generated scores for hirability and related traits. Indian transcripts receive consistently lower scores than UK transcripts, even when they were anonymized, with disparities linked to linguistic features such as sentence complexity and lexical diversity. We then perform controlled identity substitutions (varying names by gender, caste, and region) within the Indian dataset to test for name-based bias. These substitutions do not yield statistically significant effects, indicating that names alone, when isolated from other contextual signals, may not influence LLM evaluations. Our findings underscore the importance of evaluating both linguistic and social dimensions in LLM-driven evaluations and highlight the need for culturally sensitive design and accountability in AI-assisted hiring.

Problem

• Examining cultural bias in LLM-based hiring evaluations
• Assessing cross-cultural score disparities in interview assessments
• Investigating name-based bias effects in AI hiring systems

Innovation

• Cross-cultural analysis of LLM hiring evaluations
• Controlled identity substitutions for bias testing
• Linguistic feature disparities in anonymized transcripts