Problem
Research questions and friction points this paper is trying to address.
Retrieval-Augmented Generation
Position-Independent Caching
Visual Compression
Contextual Mismatch
Textual Evidence Loss
QV-PIC: Query-Aware Visual Position-Independent Caching for Efficient RAG Serving
🤖 AI Summary
This work addresses redundant computation in existing Retrieval-Augmented Generation (RAG) systems caused by repeated prefilling and the degradation in generation quality from vision-based Position-Invariant Caching (PIC), which suffers from context misalignment and detail loss. The authors propose a query-aware dual-resolution cache reuse framework that, for the first time, leverages the model’s native chat template to construct visual caches offline. During inference, it fuses low-resolution global context with high-resolution, query-relevant textual evidence and enables fine-grained recovery without recomputation through accumulated query relevance scores. This approach breaks the trade-off between visual compression and generation fidelity, achieving an average F1 improvement of 21.6 points across six tasks, outperforming optimized text-only PIC by 2.58 points, and reducing time-to-first-token (TTFT) by 83.8% compared to full prefilling—substantially narrowing the quality gap with text-based PIC.
📝 Abstract
Retrieval-Augmented Generation (RAG) repeatedly prefills identical text chunks across queries, incurring redundant computations. Position-Independent Caching (PIC) mitigates it by reusing precomputed Key-Value (KV) across positions, but its efficiency is constrained by the large volume of text tokens. Rendering text chunks as images can compress the text into fewer visual tokens, but the rendered-image PIC suffers more severe quality degradation than the text PIC. This representation-specific gap primarily arises from contextual mismatches across independently compiled caches and the loss of fine-grained textual evidence during visual compression. Existing PIC repair methods mainly address the former through selective recomputation, but they incur online computation and cannot recover lost textual details. We propose QV-PIC, a query-aware dual-resolution PIC reuse framework guided by model-native templates. Offline, QV-PIC compiles visual caches under the model's native chat-template prefix, improving PIC quality without online recomputation. Online, it preserves global context with low resolution and restores fine-grained textual evidence within a high-resolution budget by cumulative query relevance scores, retaining the efficiency benefit of visual compression. Across six tasks, QV-PIC improves average F1 by 21.6 points over vanilla rendered-image PIC, closes the gap to vanilla text PIC, and surpasses optimized text PIC by 2.58 F1 while reducing TTFT by 17.2\%. Relative to full prefill, it cuts TTFT by 83.8%.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Query-Aware Caching
Position-Independent Caching
Visual Token Compression
Dual-Resolution Reuse
Retrieval-Augmented Generation
🔎 Similar Papers
No similar papers found.
