Careful Queries, Credible Results: Teaching RAG Models Advanced Web Search Tools with Reinforcement Learning

2025-08-11

AI Summary

This work addresses two critical challenges confronting Retrieval-Augmented Generation (RAG) systems in open web environments: (1) retrieval inaccuracy caused by pervasive unreliable information, and (2) excessive noise stemming from underutilization of web tools. To tackle these issues, we propose WebFilter—a novel framework integrating source-constrained query generation with credibility-aware content filtering. WebFilter employs a behavior–outcome dual-driven reinforcement learning mechanism to jointly optimize query rewriting and retrieval filtering decisions. Technically, it unifies retrieval-augmented generation, credibility assessment, source-constrained search, and online policy optimization. Extensive experiments on both in-domain and cross-domain benchmarks demonstrate that WebFilter significantly improves retrieval precision and answer quality. Results validate its robustness and effectiveness in complex, dynamic web settings—outperforming state-of-the-art baselines across key metrics.

Technology Category

• Search and Optimization: Learning to Search
• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Algorithmic accountability and transparency on the web
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by integrating up-to-date external knowledge, yet real-world web environments present unique challenges. These limitations manifest as two key challenges: pervasive misinformation in the web environment, which introduces unreliable or misleading content that can degrade retrieval accuracy, and the underutilization of web tools, which, if effectively employed, could enhance query precision and help mitigate this noise, ultimately improving the retrieval results in RAG systems. To address these issues, we propose WebFilter, a novel RAG framework that generates source-restricted queries and filters out unreliable content. This approach combines a retrieval filtering mechanism with a behavior- and outcome-driven reward strategy, optimizing both query formulation and retrieval outcomes. Extensive experiments demonstrate that WebFilter improves answer quality and retrieval precision, outperforming existing RAG methods on both in-domain and out-of-domain benchmarks.

Problem

• Addressing misinformation in web environments for RAG systems
• Enhancing query precision by utilizing advanced web tools
• Improving retrieval accuracy and answer quality in RAG frameworks

Innovation

• WebFilter framework for reliable RAG queries
• Combines retrieval filtering with reward strategy
• Improves answer quality and retrieval precision