Balancing the Blend: An Experimental Analysis of Trade-offs in Hybrid Search

2025-08-02

AI Summary

Existing hybrid search systems lack systematic empirical analysis of trade-offs among lexical and semantic retrieval components—i.e., retrieval paradigms, fusion strategies, and re-ranking methods—leading to complex, suboptimal configurations. Method: We introduce the first benchmark framework tailored for advanced hybrid architectures, conducting systematic evaluation across 11 real-world datasets, covering four retrieval paradigms, their combinations, and re-ranking strategies. Contribution/Results: We identify a “weakest-link” effect in hybrid pipelines and propose a data-driven configuration mapping method. Crucially, we find Tensor-based Re-ranking Fusion (TRF) achieves both high efficiency and strong semantic modeling under low-resource conditions, overcoming traditional fusion bottlenecks. Experiments reveal that hybrid performance is severely constrained by imbalanced path quality; optimal configurations are highly dependent on dataset characteristics and resource constraints. TRF significantly improves the effectiveness–cost trade-off, outperforming state-of-the-art baselines across diverse settings.

Technology Category

• Search and Optimization: Algorithm Configuration
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Lexical Semantics and Morphology

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Hybrid search, the integration of lexical and semantic retrieval, has become a cornerstone of modern information retrieval systems, driven by demanding applications like Retrieval-Augmented Generation (RAG). The architectural design space for these systems is vast and complex, yet a systematic, empirical understanding of the trade-offs among their core components--retrieval paradigms, combination schemes, and re-ranking methods--is critically lacking. To address this, and informed by our experience building the Infinity open-source database, we present the first systematic benchmark of advanced hybrid search architectures. Our framework evaluates four retrieval paradigms--Full-Text Search (FTS), Sparse Vector Search (SVS), Dense Vector Search (DVS), and Tensor Search (TenS)--benchmarking their combinations and re-ranking strategies across 11 real-world datasets. Our results reveal three key findings for practitioners and researchers: (1) A "weakest link" phenomenon, where a single underperforming retrieval path can disproportionately degrade overall accuracy, highlighting the need for path-wise quality assessment before fusion. (2) A data-driven map of the performance trade-offs, demonstrating that optimal configurations depend heavily on resource constraints and data characteristics, moving beyond a one-size-fits-all approach. (3) The identification of Tensor-based Re-ranking Fusion (TRF) as a high-efficacy alternative to mainstream fusion methods, offering the semantic power of tensor search at a fraction of the computational and memory cost. Our findings offer concrete guidelines for designing the next generation of adaptive, scalable hybrid search systems while also identifying key directions for future research.

Problem

• Analyzes trade-offs in hybrid search components like retrieval and fusion
• Benchmarks hybrid search architectures across diverse real-world datasets
• Identifies optimal configurations and efficient alternatives for hybrid search

Innovation

• Systematic benchmark of hybrid search architectures
• Data-driven performance trade-offs analysis
• Tensor-based Re-ranking Fusion as efficient alternative