DeCAP: Context-Adaptive Prompt Generation for Debiasing Zero-shot Question Answering in Large Language Models

2025-03-25

AI Summary

Large language models (LLMs) exhibit biased responses to socially sensitive questions in zero-shot question answering due to inherent knowledge biases. Method: This paper proposes a context-adaptive, fine-tuning-free debiasing method that (1) introduces a novel dynamic intervention mechanism based on question ambiguity detection to identify bias-prone query patterns, and (2) integrates controllable prompt engineering with neutral semantic guidance to generate bias-mitigating prompts that enforce answer neutrality. Contribution/Results: The method requires no model fine-tuning or external bias annotations, yet achieves state-of-the-art (SOTA) performance across eight mainstream LLMs. Experiments demonstrate significant improvements in the fairness–accuracy trade-off while preserving zero-shot efficiency, outperforming existing debiasing approaches in both robustness and generalizability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Computer Vision: Bias, Fairness & Privacy

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

While Large Language Models (LLMs) excel in zero-shot Question Answering (QA), they tend to expose biases in their internal knowledge when faced with socially sensitive questions, leading to a degradation in performance. Existing zero-shot methods are efficient but fail to consider context and prevent bias propagation in the answers. To address this, we propose DeCAP, a method for debiasing LLMs using Context-Adaptive Prompt Generation. DeCAP leverages a Question Ambiguity Detection to take appropriate debiasing actions based on the context and a Neutral Answer Guidance Generation to suppress the LLMs make objective judgments about the context, minimizing the propagation of bias from their internal knowledge. Our various experiments across eight LLMs show that DeCAP achieves state-of-the-art zero-shot debiased QA performance. This demonstrates DeCAP's efficacy in enhancing the fairness and accuracy of LLMs in diverse QA settings.

Problem

• Debiasing zero-shot QA in LLMs for sensitive questions
• Adapting prompts to context for bias reduction
• Improving fairness and accuracy in diverse QA settings

Innovation

• Context-Adaptive Prompt Generation for debiasing
• Question Ambiguity Detection for context analysis
• Neutral Answer Guidance to suppress bias