LexSemBridge: Fine-Grained Dense Representation Enhancement through Token-Aware Embedding Augmentation

2025-08-25

AI Summary

To address the semantic-lexical signal mismatch of dense retrieval models in fine-grained retrieval tasks—such as keyword alignment and paragraph localization—within Retrieval-Augmented Generation (RAG), this paper proposes LexSemBridge. Our method introduces token-aware embedding enhancement that integrates three complementary latent vectors—statistical (SLR), learned (LLR), and contextual (CLR)—into dense representations in a plug-and-play manner, without modifying the backbone encoder. Leveraging element-wise interaction and vector modulation, LexSemBridge jointly strengthens discriminative dimensions while preserving semantic directionality. The framework natively supports multimodal extension to both text and vision modalities. Extensive experiments demonstrate significant performance gains across multiple semantic and fine-grained retrieval benchmarks, validating its effectiveness and generalizability. The code and models are publicly available.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

As queries in retrieval-augmented generation (RAG) pipelines powered by large language models (LLMs) become increasingly complex and diverse, dense retrieval models have demonstrated strong performance in semantic matching. Nevertheless, they often struggle with fine-grained retrieval tasks, where precise keyword alignment and span-level localization are required, even in cases with high lexical overlap that would intuitively suggest easier retrieval. To systematically evaluate this limitation, we introduce two targeted tasks, keyword retrieval and part-of-passage retrieval, designed to simulate practical fine-grained scenarios. Motivated by these observations, we propose LexSemBridge, a unified framework that enhances dense query representations through fine-grained, input-aware vector modulation. LexSemBridge constructs latent enhancement vectors from input tokens using three paradigms: Statistical (SLR), Learned (LLR), and Contextual (CLR), and integrates them with dense embeddings via element-wise interaction. Theoretically, we show that this modulation preserves the semantic direction while selectively amplifying discriminative dimensions. LexSemBridge operates as a plug-in without modifying the backbone encoder and naturally extends to both text and vision modalities. Extensive experiments across semantic and fine-grained retrieval tasks validate the effectiveness and generality of our approach. All code and models are publicly available at https://github.com/Jasaxion/LexSemBridge/

Problem

• Enhances dense retrieval for fine-grained tasks
• Addresses keyword and span-level localization challenges
• Improves semantic matching with token-aware embedding augmentation

Innovation

• Token-aware embedding augmentation for dense representations
• Element-wise interaction of latent enhancement vectors
• Plug-in framework preserving semantic directionality