The Role of the Availability Heuristic in Multiple-Choice Answering Behaviour

2026-02-19

AI Summary

This study investigates whether students can leverage the availability heuristic—selecting options that are cognitively more accessible—to improve performance on multiple-choice questions when uncertain about the correct answer. We introduce the first systematic quantification of option availability by measuring lexical frequency and conceptual popularity using large-scale corpora such as Wikipedia, and analyze its association with correct answers across three extensive question sets. Results demonstrate that correct answers consistently exhibit higher cognitive availability than distractors, a pattern robustly observed in both human-expert-authored and large language model–generated items. A strategy of always selecting the most available option yields score improvements of 13.5% to 32.9% over random guessing, confirming the practical utility of availability as an effective cue for answering under uncertainty.

Technology Category

• Reasoning under Uncertainty: Causality
• Cognitive Modeling & Cognitive Systems: Analogy
• Natural Language Processing: Question Answering

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Community question answering
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

When students are unsure of the correct answer to a multiple-choice question (MCQ), guessing is common practice. The availability heuristic, proposed by A. Tversky and D. Kahneman in 1973, suggests that the ease with which relevant instances come to mind, typically operationalised by the mere frequency of exposure, can offer a mental shortcut for problems in which the test-taker does not know the exact answer. Is simply choosing the option that comes most readily to mind a good strategy for answering MCQs? We propose a computational method of assessing the cognitive availability of MCQ options operationalised by concepts' prevalence in large corpora. The key finding, across three large question sets, is that correct answers, independently of the question stem, are significantly more available than incorrect MCQ options. Specifically, using Wikipedia as the retrieval corpus, we find that always selecting the most available option leads to scores 13.5% to 32.9% above the random-guess baseline. We further find that LLM-generated MCQ options show similar patterns of availability compared to expert-created options, despite the LLMs' frequentist nature and their training on large collections of textual data. Our findings suggest that availability should be considered in current and future work when computationally modelling student behaviour.

Problem

• availability heuristic
• multiple-choice questions
• cognitive bias
• answer guessing
• student behavior

Innovation

• availability heuristic
• multiple-choice questions
• computational modeling
• large language models
• cognitive availability