Selected work

Representative Papers

FIRE: A Comprehensive Benchmark for Financial Intelligence and Reasoning Evaluation

Feb 25, 2026

This work addresses the lack of a systematic evaluation benchmark for assessing the theoretical understanding and practical reasoning capabilities of large language models (LLMs) in the financial domain. To this end, we propose FIRE, a comprehensive evaluation benchmark that, for the first time, integrates questions from financial certification exams with real-world business scenarios. FIRE comprises 3,000 structured questions and open-ended problems, accompanied by standardized scoring rubrics. Leveraging a multidimensional capability taxonomy, we conduct a systematic evaluation of mainstream LLMs, including our in-house model XuanYuan 4.0. Our study not only reveals the current performance boundaries of existing models on financial tasks but also publicly releases the dataset and evaluation code, establishing a reliable benchmark to advance research in financial intelligence.

0 citationsRead paper

Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?

Feb 14, 2026

This study addresses the critical issue of benchmark data leakage in the evaluation of large language models (LLMs) for recommender systems, which often leads to inflated or degraded performance estimates. We systematically uncover and validate, for the first time, the memory effect of LLMs on recommendation benchmark data acquired during pretraining or fine-tuning, revealing a dual-impact mechanism: in-domain data leakage substantially inflates performance metrics, whereas out-of-domain leakage impairs recommendation accuracy. To investigate this phenomenon, we construct a hybrid corpus combining both in-domain and out-of-domain user–item interactions and conduct experiments using continual pretraining strategies. Our findings demonstrate that data leakage is a pivotal factor undermining the reliability of LLM-based recommendation evaluations, offering crucial methodological cautions and practical guidance for future research aiming to establish trustworthy assessment protocols.

0 citationsRead paper

MMFCTUB: Multi-Modal Financial Credit Table Understanding Benchmark

Jan 08, 2026arXiv.org

This work addresses the lack of high-quality multimodal benchmarks for financial credit table understanding, a field hindered by data inconsistency, high annotation costs, and misaligned evaluation metrics. To bridge this gap, we present the first multimodal benchmark comprising over 7,600 samples across five distinct table types. We introduce a weakly supervised construction approach that enforces constraint preservation and distributional consistency, alongside capability-driven question design and a mask-recovery strategy to evaluate models’ abilities in cross-table structural awareness, domain knowledge integration, and numerical reasoning. Comprehensive evaluations of leading multimodal large language models reveal their strengths and limitations in structural comprehension and logical inference, establishing a reliable benchmark and evaluation paradigm for future research.

0 citationsRead paper

FRONTIER-RevRec: A Large-scale Dataset for Reviewer Recommendation

Oct 18, 2025

Current reviewer recommendation research is hindered by the absence of large-scale, multi-disciplinary, and reproducible benchmark datasets. To address this, we introduce FRONTIER-RevRec—the largest publicly available reviewer recommendation benchmark to date—comprising 209 interdisciplinary journals, 478,000 papers, and 178,000 reviewers. Through systematic analysis, we reveal fundamental structural differences between academic and commercial recommendation: content-based methods substantially outperform collaborative filtering, and language models more effectively capture semantic alignment between papers and reviewers. Building on these insights, we propose a novel method that jointly encodes paper text and reviewer historical review records via semantically enriched representations and an optimized aggregation strategy. Our approach achieves significant improvements across multiple evaluation metrics. FRONTIER-RevRec establishes a standardized evaluation framework for automated peer review and sets a new state-of-the-art baseline for future research.

0 citationsRead paper

UFO-RL: Uncertainty-Focused Optimization for Efficient Reinforcement Learning Data Selection

May 18, 2025

To address the low data selection efficiency and high computational cost in reinforcement learning fine-tuning of large language models (LLMs), this paper proposes a single-forward uncertainty estimation framework grounded in Vygotsky’s Zone of Proximal Development (ZPD) cognitive theory. It introduces the first ZPD-inspired approach for RL data filtering, adaptively defining ZPD boundaries via learnable uncertainty modeling. By replacing multi-sample evaluation with a single forward pass, the method achieves 185× computational speedup. Integrated with policy-optimization–guided data reweighting and lightweight confidence calibration, it significantly improves selection accuracy. Experiments demonstrate that the method attains full-data performance using only 10% of training samples, delivers up to 16× end-to-end training acceleration, and markedly enhances training stability and cross-task generalization.

0 citationsRead paper

Recent publications

Latest Papers

FIRE: A Comprehensive Benchmark for Financial Intelligence and Reasoning Evaluation

Feb 25, 2026

This work addresses the lack of a systematic evaluation benchmark for assessing the theoretical understanding and practical reasoning capabilities of large language models (LLMs) in the financial domain. To this end, we propose FIRE, a comprehensive evaluation benchmark that, for the first time, integrates questions from financial certification exams with real-world business scenarios. FIRE comprises 3,000 structured questions and open-ended problems, accompanied by standardized scoring rubrics. Leveraging a multidimensional capability taxonomy, we conduct a systematic evaluation of mainstream LLMs, including our in-house model XuanYuan 4.0. Our study not only reveals the current performance boundaries of existing models on financial tasks but also publicly releases the dataset and evaluation code, establishing a reliable benchmark to advance research in financial intelligence.

0 citationsRead paper

Benchmark Leakage Trap: Can We Trust LLM-based Recommendation?

Feb 14, 2026

This study addresses the critical issue of benchmark data leakage in the evaluation of large language models (LLMs) for recommender systems, which often leads to inflated or degraded performance estimates. We systematically uncover and validate, for the first time, the memory effect of LLMs on recommendation benchmark data acquired during pretraining or fine-tuning, revealing a dual-impact mechanism: in-domain data leakage substantially inflates performance metrics, whereas out-of-domain leakage impairs recommendation accuracy. To investigate this phenomenon, we construct a hybrid corpus combining both in-domain and out-of-domain user–item interactions and conduct experiments using continual pretraining strategies. Our findings demonstrate that data leakage is a pivotal factor undermining the reliability of LLM-based recommendation evaluations, offering crucial methodological cautions and practical guidance for future research aiming to establish trustworthy assessment protocols.

0 citationsRead paper

MMFCTUB: Multi-Modal Financial Credit Table Understanding Benchmark

Jan 08, 2026arXiv.org

This work addresses the lack of high-quality multimodal benchmarks for financial credit table understanding, a field hindered by data inconsistency, high annotation costs, and misaligned evaluation metrics. To bridge this gap, we present the first multimodal benchmark comprising over 7,600 samples across five distinct table types. We introduce a weakly supervised construction approach that enforces constraint preservation and distributional consistency, alongside capability-driven question design and a mask-recovery strategy to evaluate models’ abilities in cross-table structural awareness, domain knowledge integration, and numerical reasoning. Comprehensive evaluations of leading multimodal large language models reveal their strengths and limitations in structural comprehension and logical inference, establishing a reliable benchmark and evaluation paradigm for future research.

0 citationsRead paper

FRONTIER-RevRec: A Large-scale Dataset for Reviewer Recommendation

Oct 18, 2025

Current reviewer recommendation research is hindered by the absence of large-scale, multi-disciplinary, and reproducible benchmark datasets. To address this, we introduce FRONTIER-RevRec—the largest publicly available reviewer recommendation benchmark to date—comprising 209 interdisciplinary journals, 478,000 papers, and 178,000 reviewers. Through systematic analysis, we reveal fundamental structural differences between academic and commercial recommendation: content-based methods substantially outperform collaborative filtering, and language models more effectively capture semantic alignment between papers and reviewers. Building on these insights, we propose a novel method that jointly encodes paper text and reviewer historical review records via semantically enriched representations and an optimized aggregation strategy. Our approach achieves significant improvements across multiple evaluation metrics. FRONTIER-RevRec establishes a standardized evaluation framework for automated peer review and sets a new state-of-the-art baseline for future research.

0 citationsRead paper

UFO-RL: Uncertainty-Focused Optimization for Efficient Reinforcement Learning Data Selection

May 18, 2025

To address the low data selection efficiency and high computational cost in reinforcement learning fine-tuning of large language models (LLMs), this paper proposes a single-forward uncertainty estimation framework grounded in Vygotsky’s Zone of Proximal Development (ZPD) cognitive theory. It introduces the first ZPD-inspired approach for RL data filtering, adaptively defining ZPD boundaries via learnable uncertainty modeling. By replacing multi-sample evaluation with a single forward pass, the method achieves 185× computational speedup. Integrated with policy-optimization–guided data reweighting and lightweight confidence calibration, it significantly improves selection accuracy. Experiments demonstrate that the method attains full-data performance using only 10% of training samples, delivers up to 16× end-to-end training acceleration, and markedly enhances training stability and cross-task generalization.

0 citationsRead paper