Query-aware Routing for Filtered Approximate Nearest Neighbors Search

2026-06-18

AI Summary

This work addresses the challenge of achieving adaptive performance in filtered approximate nearest neighbor (Filtered ANN) search, where existing methods struggle to uniformly meet diverse query-specific requirements. The authors propose a query-aware dynamic routing framework that employs a lightweight regression model to accurately predict the recall of each candidate algorithm for a given query using only three key features. By integrating these predictions with an offline-constructed performance benchmark table, the framework dynamically selects the strategy that optimally balances recall and queries per second (QPS). This approach introduces, for the first time, a query-driven routing mechanism that achieves state-of-the-art recall–QPS trade-offs across five unseen datasets while incurring negligible inference overhead.

Abstract

Filtered ANN search, which combines vector similarity with attribute predicates, is a core primitive in modern vector databases and retrieval-augmented generation. We benchmark all major categorical filtered ANN methods across multiple datasets under three predicates and find that no single method dominates. Moreover, even within a single dataset and predicate type, the best method for a query can vary. Therefore, we propose a query-aware routing framework. A lightweight ML model predicts each candidate method's recall on the query, and the router consults an offline benchmark table that maps every method and parameter setting to its measured recall and QPS, then selects the method with the best recall--QPS trade-off. Our ablation study narrows 22 candidate features to a minimal set of three and we adopt regression rather than classification as the prediction target to sharpen accuracy. Our model is trained on six real-world datasets and applied to five unseen validation datasets. The final result shows that our router achieves state-of-the-art recall and QPS balance across all five validation datasets compared to existing filtered ANN baselines, while incurring negligible latency overhead.

Problem

• Filtered ANN
• query-aware routing
• vector similarity
• attribute predicates
• recall-QPS trade-off

Innovation

• query-aware routing
• filtered approximate nearest neighbors
• recall-QPS trade-off
• lightweight ML model
• vector databases