Beyond the Resumé: A Rubric-Aware Automatic Interview System for Information Elicitation

2026-03-02

AI Summary

Traditional hiring relies heavily on expert-led interviews, which are costly, while automated resume-based screening offers limited information and struggles to accurately assess candidate competencies. This work proposes a large language model (LLM)-driven simulated interview framework that, for the first time, integrates structured scoring rubrics with a Bayesian belief updating mechanism to dynamically and calibratably infer candidates’ role-relevant latent abilities. The system actively elicits fine-grained evidence through interactive dialogue, enabling belief estimates to effectively converge toward true competency levels as the interview progresses. Experimental results demonstrate the efficacy of the proposed approach, and the code, dataset, and a demonstration system have been publicly released.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Effective hiring is integral to the success of an organisation, but it is very challenging to find the most suitable candidates because expert evaluation (e.g.\ interviews conducted by a technical manager) are expensive to deploy at scale. Therefore, automated resume scoring and other applicant-screening methods are increasingly used to coarsely filter candidates, making decisions on limited information. We propose that large language models (LLMs) can play the role of subject matter experts to cost-effectively elicit information from each candidate that is nuanced and role-specific, thereby improving the quality of early-stage hiring decisions. We present a system that leverages an LLM interviewer to update belief over an applicant's rubric-oriented latent traits in a calibrated way. We evaluate our system on simulated interviews and show that belief converges towards the simulated applicants' artificially-constructed latent ability levels. We release code, a modest dataset of public-domain/anonymised resumes, belief calibration tests, and simulated interviews, at \href{https://github.com/mbzuai-nlp/beyond-the-resume}{https://github.com/mbzuai-nlp/beyond-the-resume}. Our demo is available at \href{https://btr.hstu.net}{https://btr.hstu.net}.

Problem

• automated hiring
• candidate screening
• information elicitation
• early-stage hiring decisions
• resume-based filtering

Innovation

• large language models
• rubric-aware interviewing
• belief calibration
• automated hiring
• latent trait estimation