SQLite is Enough. Lexical, Semantic, and Hybrid Search with scrydb

2026-08-25

AI Summary

本文介绍了一个名为scrydb的Python库，它通过利用SQLite的功能实现了词法、语义和混合搜索，为信息检索任务提供了一种轻量级解决方案。

Abstract

This work introduces scrydb, a Python library that enables lexical, semantic, and hybrid search within SQLite. For lexical search, scrydb leverages SQLite's full-text search extension FTS5. Semantic search builds on sqlite-vec, a SQLite extension for vector search. Furthermore, the library allows users to rerank and fuse retrieval results to combine both lexical and semantic approaches, providing a lightweight solution for downstream tasks in information retrieval (IR) or agentic search. We evaluate scrydb on various IR benchmark datasets and demonstrate its effectiveness in text retrieval based on keyword matching, semantic similarity, and rank fusion. In addition, we provide insights into query latency and the trade-off between efficiency and effectiveness. scrydb is available under the MIT license.

Problem

• lexical search
• semantic search
• hybrid search
• information retrieval

Innovation

• lexical search
• semantic search
• hybrid search
• SQLite
• vector search