Work Sharing and Offloading for Efficient Approximate Threshold-based Vector Join

2026-03-17

AI Summary

This work addresses the inefficiency of existing vector join methods in threshold queries, which suffer from redundant index traversals and excessive distance computations. To overcome these limitations, the authors propose a unified framework that integrates three key innovations: a soft work-sharing mechanism, a merged graph index that jointly embeds query and data vectors, and an adaptive hybrid search strategy tailored for out-of-distribution queries. Evaluated on eight benchmark datasets, the proposed approach consistently outperforms state-of-the-art methods, achieving substantially lower computational overhead while maintaining high recall. The results demonstrate a superior trade-off between efficiency and accuracy for approximate threshold vector joins.

Technology Category

• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Vector joins - finding all vector pairs between a set of query and data vectors whose distances are below a given threshold - are fundamental to modern vector and vector-relational database systems that power multimodal retrieval and semantic analytics. Existing state-of-the-art approach exploits work sharing among similar queries but still suffers from redundant index traversals and excessive distance computations. We propose a unified framework for efficient approximate vector joins that (1) introduces soft work sharing to reuse traversal results beyond the join results of previous queries, (2) builds a merged index over both query and data vectors to further speedup graph explorations, and (3) improves robustness for out-of-distribution queries through an adaptive hybrid search strategy. Experiments on eight datasets demonstrate substantial improvements in efficiency-recall trade-off over the state of the art.

Problem

• vector join
• threshold-based
• work sharing
• index traversal
• distance computation

Innovation

• soft work sharing
• merged index
• adaptive hybrid search
• approximate vector join
• graph exploration