Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization

2026-05-12

AI Summary

This work addresses the sensitivity of large language models (LLMs) to input order, which undermines fairness and reliability in tasks such as in-context learning and retrieval-augmented generation (RAG). To mitigate this ordering bias, we propose Dual-Group Advantage Optimization (DGAO), the first approach leveraging reinforcement learning to jointly optimize intra-group accuracy advantage and inter-group stability advantage, thereby enhancing both correctness and order robustness of model outputs. We introduce two novel evaluation metrics—consistency rate and overconfidence rate—to expose the illusory stability of existing methods. Experimental results demonstrate that DGAO significantly outperforms baseline approaches across RAG, mathematical reasoning, and classification tasks, achieving improved order fairness without compromising accuracy.

Abstract

Large Language Models (LLMs) suffer from order bias, where their performance is affected by the arrangement order of input elements. This unfairness limits the model's applications in scenarios such as in-context learning and Retrieval-Augmented Generation (RAG). Recent studies attempt to obtain optimal or suboptimal arrangements based on statistical results or using dataset-based search, but these methods increase inference overhead while leaving the model's inherent order bias unresolved. Other studies mitigate order sensitivity through supervised fine-tuning using augmented training sets with multiple order variants, but often at the cost of accuracy, trapping the model in consistent yet incorrect hallucinations. In this paper, we propose \textbf{D}ual \textbf{G}roup \textbf{A}dvantage \textbf{O}ptimization (\textbf{DGAO}), which aims to improve model accuracy and order stability simultaneously. DGAO calculates and balances intra-group relative accuracy advantage and inter-group relative stability advantage, rewarding the policy model for generating order-stable and correct outputs while penalizing order-sensitive or incorrect responses. This marks the first time reinforcement learning has been used to mitigate LLMs' order sensitivity. We also propose two new metrics, Consistency Rate and Overconfidence Rate, to reveal the pseudo-stability of previous methods and guide more comprehensive evaluation. Extensive experiments demonstrate that DGAO achieves superior order fairness while improving performance on RAG, mathematical reasoning, and classification tasks. Our code is available at: https://github.com/Hyalinesky/DGAO.

Problem

• order bias
• order sensitivity
• large language models
• fairness
• in-context learning

Innovation

• Order Fairness
• Reinforcement Learning
• Dual Group Advantage Optimization
• LLM Robustness
• Sequence Sensitivity