vstash: Local-First Hybrid Retrieval with Adaptive Fusion for LLM Agents

2026-04-16

AI Summary

This work addresses the limitations of single-retrieval approaches in semantic and keyword matching for large language model agents operating in local-first settings. We propose a hybrid retrieval framework that constructs a local document memory system using SQLite, integrating vector similarity search (via sqlite-vec) with full-text keyword matching (FTS5). The system employs an adaptive Reciprocal Rank Fusion (RRF) strategy combined with per-query IDF weighting. Innovatively, we introduce a self-supervised embedding fine-tuning method based on retrieval disagreement, optimizing BGE-small with MultipleNegativesRankingLoss. The framework also incorporates integrity verification, version control, and diagnostic tools. Experiments demonstrate up to a 19.5% improvement in NDCG@10 across five BEIR datasets, with our 33M-parameter model matching or surpassing the 110M-parameter ColBERTv2 and BGE-base on three datasets, while achieving a median retrieval latency of only 20.9ms over 50,000 text chunks.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Local Search
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We present **vstash**, a local-first document memory system that combines vector similarity search with full-text keyword matching via Reciprocal Rank Fusion (RRF) and adaptive per-query IDF weighting. All data resides in a single SQLite file using sqlite-vec for approximate nearest neighbor search and FTS5 for keyword matching. We make four primary contributions. **(1)** Self-supervised embedding refinement via hybrid retrieval disagreement: across 753 BEIR queries on SciFact, NFCorpus, and FiQA, 74.5% produce top-10 disagreement between vector-heavy (vec=0.95, fts=0.05) and FTS-heavy (vec=0.05, fts=0.95) search (per-dataset rates 63.4% / 73.4% / 86.7%, Section 5.2), providing a free training signal without human labels. Fine-tuning BGE-small (33M params) with MultipleNegativesRankingLoss on 76K disagreement triples improves NDCG@10 on all 5 BEIR datasets (up to +19.5% on NFCorpus vs. BGE-small base RRF, Table 6). On 3 of 5 datasets, under different preprocessing, the tuned 33M-parameter pipeline matches or exceeds published ColBERTv2 results (110M params) and an untrained BGE-base (110M); on FiQA and ArguAna it underperforms ColBERTv2 (Section 5.5). **(2)** Adaptive RRF with per-query IDF weighting improves NDCG@10 on all 5 BEIR datasets versus fixed weights (up to +21.4% on ArguAna), achieving 0.7263 on SciFact with BGE-small. **(3)** A negative result on post-RRF scoring: frequency+decay, history-augmented recall, and cross-encoder reranking all failed to improve NDCG. **(4)** A production-grade substrate with integrity checking, schema versioning, ranking diagnostics, and a distance-based relevance signal validated on 50,425 relevance-judged queries across the 5 BEIR datasets. Search latency remains 20.9 ms median at 50K chunks with stable NDCG. The fine-tuned model is published as `Stffens/bge-small-rrf-v2` on HuggingFace. All code, data, and experiments are open-source.

Problem

• hybrid retrieval
• local-first
• adaptive fusion
• LLM agents
• unsupervised training signal

Innovation

• adaptive RRF
• self-supervised embedding refinement
• local-first retrieval
• hybrid retrieval disagreement
• IDF-weighted fusion