Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring

2026-04-21

AI Summary

This study reveals that large language models (LLMs) introduce bias in evaluative language when generating résumé summaries due to racial and gender cues implicitly associated with candidate names, thereby compromising hiring fairness. Through a large-scale controlled experiment analyzing nearly one million summaries produced by four leading LLMs under systematic name perturbations, the work innovatively decomposes generated content into factual and evaluative components. It identifies and quantifies, for the first time, a symmetric instability in the distributional tails of evaluative language—a subtle bias that evades conventional fairness audits yet induces harmful decision-making disparities in LLM-driven automated hiring systems. This effect is particularly pronounced in open-source models, challenging prevailing paradigms of algorithmic fairness evaluation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Bias, Fairness & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Large language models for search
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Research has documented LLMs' name-based bias in hiring and salary recommendations. In this paper, we instead consider a setting where LLMs generate candidate summaries for downstream assessment. In a large-scale controlled study, we analyze nearly one million resume summaries produced by 4 models under systematic race-gender name perturbations, using synthetic resumes and real-world job postings. By decomposing each summary into resume-grounded factual content and evaluative framing, we find that factual content remains largely stable, while evaluative language exhibits subtle name-conditioned variation concentrated in the extremes of the distribution, especially in open-source models. Our hiring simulation demonstrates how evaluative summary transforms directional harm into symmetric instability that might evade conventional fairness audit, highlighting a potential pathway for LLM-to-LLM automation bias.

Problem

• name-conditioned bias
• evaluative framing
• LLM-based hiring
• fairness audit
• automation bias

Innovation

• evaluative framing
• name-conditioned bias
• LLM-based hiring
• distributional tails
• automation bias