PSC: Extending Context Window of Large Language Models via Phase Shift Calibration
To address the challenge of suboptimal preset frequency scaling factors in RoPE-based context window extension—where the search space grows exponentially—this paper proposes a lightweight Phase Shift Calibration (PSC) module. PSC introduces, for the first time, a learnable phase shift mechanism that dynamically calibrates preconfigured frequency scaling (e.g., in PI, YaRN, and LongRoPE) without altering the original RoPE architecture or requiring model retraining. Its core components include differentiable phase calibration, a lightweight linear projection, and context-length-adaptive initialization. Experiments demonstrate that PSC consistently reduces perplexity on long-context benchmarks (16K–64K tokens), exhibits robust cross-model (Llama, Qwen) and cross-task (QA, long-document reasoning) performance, and delivers increasingly substantial gains as context length grows. Overall, PSC significantly enhances the robustness and plug-and-play applicability of existing RoPE extension methods.