Position Bias Correction is Insufficient for One-Pass Attention Sorting

2026-06-26

AI Summary

This work addresses the positional bias in long-context language models, where information at intermediate positions is underutilized. Existing attention sorting methods mitigate this bias through multi-pass re-ranking, but incur high deployment costs. The paper proposes a single-pass debiased attention sorting approach that estimates the positional bias curve from low-attention documents and corrects the original attention scores via subtraction or division, enabling effective one-round reranking. Systematic evaluation on YaRN-Llama-2-7b-64k shows that positional bias correction alone improves accuracy by 8.67 percentage points—yet accounts for only 37% of the performance gap closed by iterative methods—revealing that the benefits of repeated reranking extend beyond mere bias correction.

Abstract

Long-context language models suffer from position bias, where information in middle positions is underutilized. Attention Sorting addresses this by iteratively reordering documents based on attention patterns, but its multiple sort-and-generate cycles increase deployment cost. We hypothesize that position bias is the primary bottleneck and propose Debiased One-Pass Attention Sorting, which estimates a per-prompt position-bias curve from the low-attention majority of documents and uses it to correct raw attention scores (via subtraction or division) to enable single-pass sorting. Our experiments on two models refute this hypothesis in the tested setting: on LLaMA-2-7B-32K-Instruct, debiasing produces identical results to uncalibrated single-pass sorting (94.83\% containment accuracy), while on YaRN-Llama-2-7b-64k, debiasing improves accuracy by 8.67 percentage points but remains 14.84pp behind iterative sorting, closing only 37\% of the gap. These results suggest that position-bias correction is insufficient to match iterative sorting, and that repeated reordering provides additional benefits beyond bias correction.

Problem

• position bias
• attention sorting
• long-context language models
• one-pass sorting
• bias correction

Innovation

• Position Bias Correction
• One-Pass Attention Sorting
• Long-context Language Models
• Attention Reordering
• Debiased Attention