Better Together: Quantifying the Benefits of AI-Assisted Recruitment

2025-07-08

AI Summary

Despite growing adoption of AI in hiring, causal evidence on its impact on efficiency and selection quality—and potential biases—remains scarce. Method: We conducted the first randomized controlled trial evaluating an AI-augmented recruitment process, deploying an NLP-powered structured video interview system integrated with human reviewers, and compared it against conventional resume screening. Contribution/Results: AI assistance increased final interview pass rates by 20 percentage points and improved five-month employment probability by 5.9 percentage points, providing the first causal evidence of AI’s efficacy in recruitment. However, we uncovered systematic algorithmic bias: the AI disproportionately favored younger, less-experienced candidates. These findings offer critical empirical grounding for algorithmic fairness governance and human-AI collaborative hiring design, highlighting both performance gains and equity risks inherent in AI-driven selection systems.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Humans and AI: AI for Accessibility
• Machine Learning: Ethics, Bias, and Fairness

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Artificial intelligence (AI) is increasingly used in recruitment, yet empirical evidence quantifying its impact on hiring efficiency and candidate selection remains limited. We randomly assign 37,000 applicants for a junior-developer position to either a traditional recruitment process (resume screening followed by human selection) or an AI-assisted recruitment pipeline incorporating an initial AI-driven structured video interview before human evaluation. Candidates advancing from either track faced the same final-stage human interview, with interviewers blind to the earlier selection method. In the AI-assisted pipeline, 54% of candidates passed the final interview compared with 34% from the traditional pipeline, yielding an average treatment effect of 20 percentage points (SE 12 pp.). Five months later, we collected LinkedIn profiles of top applicants from both groups and found that 18% (SE 1.1%) of applicants from the traditional track found new jobs compared with 23% (SE 2.3%) from the AI group, resulting in a 5.9 pp. (SE 2.6 pp.) difference in the probability of finding new employment between groups. The AI system tended to select younger applicants with less experience and fewer advanced credentials. We analyze AI-generated interview transcripts to examine the selection criteria and conversational dynamics. Our findings contribute to understanding how AI technologies affect decision making in recruitment and talent acquisition while highlighting some of their potential implications.

Problem

• Quantify AI impact on hiring efficiency and candidate selection
• Compare AI-assisted vs traditional recruitment process outcomes
• Analyze AI selection criteria and potential employment implications

Innovation

• AI-driven structured video interview screening
• Randomized comparison with traditional recruitment
• LinkedIn profile analysis for outcome validation