FinCPRG: A Bidirectional Generation Pipeline for Hierarchical Queries and Rich Relevance in Financial Chinese Passage Retrieval

2025-08-04

AI Summary

Existing Chinese financial paragraph retrieval methods suffer from weak cross-document query modeling capability and low-quality relevance annotations. To address these bottlenecks, this paper proposes a bidirectional query generation framework that hierarchically constructs queries by integrating bottom-up strategies (sentence- and paragraph-level LLM generation) with top-down strategies (multi-document title clustering guided by industry, topic, and temporal dimensions). We further introduce an indirect positive sample mining mechanism to enhance annotation robustness. Leveraging 1,300 Chinese financial research reports, we construct FinCPRG—the first fine-grained paragraph retrieval dataset tailored for the financial domain—featuring three-tier queries (cross-document, intra-document, and hybrid) and dual-label relevance annotations (direct and indirect). Experiments demonstrate that FinCPRG significantly improves both supervised training efficacy and zero-shot transfer performance of retrieval models, achieving state-of-the-art results across multiple benchmarks.

Technology Category

• Data Mining & Knowledge Management: Intelligent Query Processing
• Natural Language Processing: Discourse, Pragmatics & Argument Mining
• Computer Vision: Image and Video Retrieval

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Web Mining and Content Analysis: Community question answering

Abstract

In recent years, large language models (LLMs) have demonstrated significant potential in constructing passage retrieval datasets. However, existing methods still face limitations in expressing cross-doc query needs and controlling annotation quality. To address these issues, this paper proposes a bidirectional generation pipeline, which aims to generate 3-level hierarchical queries for both intra-doc and cross-doc scenarios and mine additional relevance labels on top of direct mapping annotation. The pipeline introduces two query generation methods: bottom-up from single-doc text and top-down from multi-doc titles. The bottom-up method uses LLMs to disassemble and generate structured queries at both sentence-level and passage-level simultaneously from intra-doc passages. The top-down approach incorporates three key financial elements--industry, topic, and time--to divide report titles into clusters and prompts LLMs to generate topic-level queries from each cluster. For relevance annotation, our pipeline not only relies on direct mapping annotation from the generation relationship but also implements an indirect positives mining method to enrich the relevant query-passage pairs. Using this pipeline, we constructed a Financial Passage Retrieval Generated dataset (FinCPRG) from almost 1.3k Chinese financial research reports, which includes hierarchical queries and rich relevance labels. Through evaluations of mined relevance labels, benchmarking and training experiments, we assessed the quality of FinCPRG and validated its effectiveness as a passage retrieval dataset for both training and benchmarking.

Problem

• Generates hierarchical queries for financial document retrieval
• Improves annotation quality with indirect relevance mining
• Addresses cross-doc query needs in financial research

Innovation

• Bidirectional generation pipeline for hierarchical queries
• Bottom-up and top-down query generation methods
• Indirect positives mining for relevance annotation