Option-ID Based Elimination For Multiple Choice Questions

2025-01-25

AI Summary

Large language models (LLMs) suffer from low elimination efficiency and high computational overhead in multiple-choice reasoning. Method: This paper proposes a novel serialization-based elimination paradigm anchored on option IDs, constructing a lightweight, sequence-driven elimination mechanism that avoids explicit error detection or per-option scoring—thereby significantly reducing inference latency and computational cost. Integrated with zero-shot/few-shot prompting, ID encoding, sequential elimination strategies, and bias-mitigating ensemble techniques, the approach operates without requiring model fine-tuning or auxiliary classifiers. Contribution/Results: Evaluated across seven benchmark datasets and ten mainstream LLMs, it consistently outperforms existing elimination methods, demonstrating strong generalization in few-shot and bias-sensitive settings. Its core innovation lies in the first systematic use of option IDs as structured, sequential representations for reasoning—enabling efficient, scalable, and low-overhead multiple-choice problem solving.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Multiple choice questions (MCQs) are a common and important task for evaluating large language models (LLMs). Based on common strategies humans use when answering MCQs, the process of elimination has been proposed as an effective problem-solving method. Existing methods to the process of elimination generally fall into two categories: one involves having the model directly select the incorrect answer, while the other involves scoring the options. However, both methods incur high computational costs and often perform worse than methods that answer based on option ID. To address this issue, this paper proposes a process of elimination based on option ID. We select 10 LLMs and conduct zero-shot experiments on 7 different datasets. The experimental results demonstrate that our method significantly improves the model's performance. Further analysis reveals that the sequential elimination strategy can effectively enhance the model's reasoning ability. Additionally, we find that sequential elimination is also applicable to few-shot settings and can be combined with debias methods to further improve model performance.

Problem

• Large Language Models
• Multiple Choice Questions
• Resource Efficiency

Innovation

• Improved Elimination Strategy
• Sequential Error Option Exclusion
• Bias Reduction Method