RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention
This work addresses the challenge of scaling large language models to ultra-long contexts, where the quadratic complexity of full self-attention becomes prohibitive. The authors propose RIS-Kernel, a general-purpose architecture that reduces self-attention complexity from O(N²) to O(N log N) through sparse random geometric sampling, without modifying model weights or requiring GPU acceleration. Notably, this approach pioneers the use of sparse attention as a regularization mechanism, achieving higher inference accuracy even at extremely low densities (e.g., 1%) while circumventing memory bottlenecks. Experiments demonstrate that RIS-Stochastic attains 75.00% accuracy on 32K-context tasks, outperforming dense baselines, and improves over zero-context baselines by 14.06 percentage points on 65K-context tasks—all efficiently executable on standard CPU servers with only 16–128 GB of memory.