🤖 AI Summary
This work addresses the critical limitation of existing semantic caching approaches, which neglect the temporal dynamics of evidence freshness in open-web settings, often yielding outdated results. To overcome this, we propose FreshCache, the first framework to formulate cache reuse as a risk-constrained sequential inference problem. FreshCache integrates an exponential decay model with a multilayer perceptron (MLP) to predict the probability of cache entry staleness and employs a three-tier risk-gating mechanism for fine-grained freshness control, enabling cache entries to degrade gracefully over time rather than fail in a binary manner. We introduce FreshCache-Bench, a novel benchmark based on real-world web snapshots, to evaluate freshness-aware caching. Experiments demonstrate that within a 24-hour window, FreshCache_MLP reduces search API calls by 97% while maintaining a hash-level staleness error rate of merely 0.1%, with only 0.034% of errors materially affecting answer correctness—substantially outperforming current methods.
📝 Abstract
Semantic caching reduces the latency and cost of retrieval-augmented generation (RAG) by serving cached answers to semantically similar queries, but most existing methods do not model the time-varying freshness of open-web evidence. We present FreshCache, a three-tier semantic cache that treats cache reuse as a risk-constrained temporal inference problem: before approving a cache hit, FreshCache estimates the probability that the cached result is stale using a fitted exponential decay model enhanced by a learned MLP, and approves reuse only when that probability falls below a per-tier error budget across answers (epsilon = 0.10), URL lists (epsilon = 0.20), and page content (epsilon = 0.35). This allows the system to degrade gracefully as entries age rather than forcing a binary choice between a stale hit and a full pipeline execution. We introduce FreshCache-Bench, a benchmark of 8,072 base queries across five freshness classes with ground truth staleness labels drawn from real web snapshots at 1, 12, 24 hours, and 7 days after a baseline crawl, expanded to 31,201 queries via paraphrase generation. At the 24-hour evaluation window, FreshCache_MLP achieves 97% search API savings at 0.1% hash-based stale error, and an LLM-judge evaluation on 396 confirmed change pairs shows that only 34.3% of detected content changes actually affect answer correctness, placing true answer-affecting stale error at approximately 0.034%. The rule-based FreshCache achieves 98% search savings at 3.3% stale error under a temporal holdout calibration, outperforming SemanticTTL (14.9% stale, 72% saved), vCache (7.2% stale, 47% saved), and SCALM (5.2% stale, 96% saved). Ablations show the temporal risk gate accounts for an 11.6 point reduction in stale error over similarity-only reuse, and the learned MLP reduces stale error a further 3.2 points over the rule-based model.