Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations

2025-02-18

AI Summary

Strategic use of large language models (LLMs) by job applicants to embellish resumes exacerbates hiring bias and undermines fairness, particularly when LLM access is unequal across demographic groups. Method: We propose a “dual-vote” fair classification framework: for each original resume, we generate a controlled LLM-rewritten version (using fine-tuned Llama-3) and jointly classify both inputs. Our approach integrates strategic classification theory, constrained optimization maximizing true positive rate (TPR) under zero false-positive constraints, and calibrated rewriting. Contribution/Results: We theoretically prove that the dual-vote mechanism simultaneously improves accuracy and inter-group fairness, and generalize it to an *n*-vote scheme, establishing convergence to a population-invariant stable decision. Experiments on real-world resume datasets show that our method significantly mitigates LLM-access-induced hiring bias, boosts accuracy by 12.7%, and strictly satisfies the zero-false-positive constraint.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Voting
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In an era of increasingly capable foundation models, job seekers are turning to generative AI tools to enhance their application materials. However, unequal access to and knowledge about generative AI tools can harm both employers and candidates by reducing the accuracy of hiring decisions and giving some candidates an unfair advantage. To address these challenges, we introduce a new variant of the strategic classification framework tailored to manipulations performed using large language models, accommodating varying levels of manipulations and stochastic outcomes. We propose a ``two-ticket'' scheme, where the hiring algorithm applies an additional manipulation to each submitted resume and considers this manipulated version together with the original submitted resume. We establish theoretical guarantees for this scheme, showing improvements for both the fairness and accuracy of hiring decisions when the true positive rate is maximized subject to a no false positives constraint. We further generalize this approach to an $n$-ticket scheme and prove that hiring outcomes converge to a fixed, group-independent decision, eliminating disparities arising from differential LLM access. Finally, we empirically validate our framework and the performance of our two-ticket scheme on real resumes using an open-source resume screening tool.

Problem

• Addresses unfair hiring via LLM manipulations
• Introduces two-ticket scheme for fairness
• Ensures accuracy in hiring decisions

Innovation

• Two-ticket resume manipulation
• Strategic classification framework
• Fairness and accuracy guarantees