Selected work

Representative Papers

RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction

Aug 06, 2026

This work addresses the challenge that generative reward models, due to their comparative outputs, are incompatible with the scalar rewards required by reinforcement learning, thereby hindering effective training of large language models. To overcome this limitation, the paper proposes a Ranking-based Reward Construction (RRC) method, which innovatively introduces self-competitive ranking and anchor-guided ranking strategies to transform relative preference rankings into scalar reward signals suitable for reinforcement learning. By circumventing the constraints of conventional scalar reward formulation, RRC achieves substantially improved training performance on open-ended dialogue and reasoning benchmarks, consistently outperforming existing reward modeling approaches.

0 citationsRead paper

D$^2$F-ReAG: Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation

Aug 05, 2026

This work addresses the limitations of existing retrieval-augmented generation methods in multi-hop question answering, which often struggle to balance efficiency and accuracy due to the absence of dynamic question decomposition and effective reasoning filtering. The authors propose a novel reasoning-enhanced generation paradigm that dynamically decides whether to decompose a question based on the reliability of root-level reasoning, and subsequently refines the final answer using verification results from sub-questions. By integrating a dynamic reasoning depth control mechanism that combines reliability assessment, logical decomposition, and retrieval-augmented generation, the method achieves significant performance gains over state-of-the-art approaches on three established benchmarks—HotpotQA, 2WikiMultihopQA, and MuSiQue—demonstrating marked improvements in accuracy for complex multi-hop reasoning tasks.

0 citationsRead paper

FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models

Jul 27, 2026

This work addresses the challenges in existing on-policy distillation based on KL divergence (KL-OPD) for flow models, which suffers from temporal supervision misalignment due to auxiliary SDE transition kernels and is further hindered by sparse rewards and exposure bias. To overcome these limitations, the authors propose FlowCTS, a novel approach that introduces continuous trajectory supervision into the on-policy distillation framework for flow models. By continuously aligning student and reference trajectories originating from the same state, FlowCTS leverages the integral relationship between trajectories and the velocity field to construct a time-weighted upper bound for velocity matching, which is then discretized into an optimizable objective. This method effectively mitigates temporal supervision mismatch and accommodates multiple reference trajectories, achieving significant improvements over KL-OPD and hybrid reward reinforcement learning baselines on GenEval (0.90→0.93), OCR (0.90→0.92), and PickScore (22.75→23.06).

0 citationsRead paper

ToFu: A White-Box, Token-Efficient Agent Harness for Researchers

Jul 13, 2026

Existing agent systems struggle to integrate deeply into scientific workflows due to low token efficiency, high costs, opaque black-box designs, and limited customizability. To address these limitations, this work proposes a white-box agent framework tailored for researchers, which supports codebase reading, file editing, command execution, and integration with development tools, serving simultaneously as a practical assistant and a researchable artifact. Built upon large language models, the framework employs a modular orchestration architecture that enables tool invocation, behavior monitoring, and logic modification. It offers multilingual support, local deployment capability, and full inspectability. While maintaining strong benchmark performance and a seamless user experience, the system significantly reduces inference costs, making it suitable for privacy-sensitive scenarios and providing an open-source (MIT-licensed), reproducible experimental platform for agent mechanism research.

0 citationsRead paper

SPRI: SVD-Partitioned Residual Initialization for Data-Constrained MoE Upcycling

Jun 15, 2026

This work addresses the challenge in data-constrained supervised fine-tuning of Mixture-of-Experts (MoE) models, where existing adaptation methods often induce expert homogenization or excessively perturb pretrained parameters, failing to balance architectural reuse with diversity. To overcome this, we propose SVD-Partitioned Residual Initialization (SPRI), which— for the first time— decomposes the pretrained feed-forward network (FFN) weights via singular value decomposition (SVD) and allocates the resulting residual components across distinct experts. This approach preserves the spectral structure of the original weights while introducing controlled diversity, further enhanced by a two-stage training strategy to improve stability. Evaluated on 15 English-to-many translation directions in CoVoST2, SPRI achieves average gains of 2.58 BLEU and 3.32 COMET points over fully fine-tuned dense models, and outperforms the previous best MoE method by 3.39 BLEU and 4.34 COMET points.

0 citationsRead paper

Recent publications

Latest Papers

RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction

Aug 06, 2026

This work addresses the challenge that generative reward models, due to their comparative outputs, are incompatible with the scalar rewards required by reinforcement learning, thereby hindering effective training of large language models. To overcome this limitation, the paper proposes a Ranking-based Reward Construction (RRC) method, which innovatively introduces self-competitive ranking and anchor-guided ranking strategies to transform relative preference rankings into scalar reward signals suitable for reinforcement learning. By circumventing the constraints of conventional scalar reward formulation, RRC achieves substantially improved training performance on open-ended dialogue and reasoning benchmarks, consistently outperforming existing reward modeling approaches.

0 citationsRead paper

D$^2$F-ReAG: Dynamic Decomposition and Filtering for Multi-Hop Reasoning-Augmented Generation

Aug 05, 2026

This work addresses the limitations of existing retrieval-augmented generation methods in multi-hop question answering, which often struggle to balance efficiency and accuracy due to the absence of dynamic question decomposition and effective reasoning filtering. The authors propose a novel reasoning-enhanced generation paradigm that dynamically decides whether to decompose a question based on the reliability of root-level reasoning, and subsequently refines the final answer using verification results from sub-questions. By integrating a dynamic reasoning depth control mechanism that combines reliability assessment, logical decomposition, and retrieval-augmented generation, the method achieves significant performance gains over state-of-the-art approaches on three established benchmarks—HotpotQA, 2WikiMultihopQA, and MuSiQue—demonstrating marked improvements in accuracy for complex multi-hop reasoning tasks.

0 citationsRead paper

FlowCTS: On-policy Continuous Trajectory Supervision of Flow Models

Jul 27, 2026

This work addresses the challenges in existing on-policy distillation based on KL divergence (KL-OPD) for flow models, which suffers from temporal supervision misalignment due to auxiliary SDE transition kernels and is further hindered by sparse rewards and exposure bias. To overcome these limitations, the authors propose FlowCTS, a novel approach that introduces continuous trajectory supervision into the on-policy distillation framework for flow models. By continuously aligning student and reference trajectories originating from the same state, FlowCTS leverages the integral relationship between trajectories and the velocity field to construct a time-weighted upper bound for velocity matching, which is then discretized into an optimizable objective. This method effectively mitigates temporal supervision mismatch and accommodates multiple reference trajectories, achieving significant improvements over KL-OPD and hybrid reward reinforcement learning baselines on GenEval (0.90→0.93), OCR (0.90→0.92), and PickScore (22.75→23.06).

0 citationsRead paper

ToFu: A White-Box, Token-Efficient Agent Harness for Researchers

Jul 13, 2026

Existing agent systems struggle to integrate deeply into scientific workflows due to low token efficiency, high costs, opaque black-box designs, and limited customizability. To address these limitations, this work proposes a white-box agent framework tailored for researchers, which supports codebase reading, file editing, command execution, and integration with development tools, serving simultaneously as a practical assistant and a researchable artifact. Built upon large language models, the framework employs a modular orchestration architecture that enables tool invocation, behavior monitoring, and logic modification. It offers multilingual support, local deployment capability, and full inspectability. While maintaining strong benchmark performance and a seamless user experience, the system significantly reduces inference costs, making it suitable for privacy-sensitive scenarios and providing an open-source (MIT-licensed), reproducible experimental platform for agent mechanism research.

0 citationsRead paper

SPRI: SVD-Partitioned Residual Initialization for Data-Constrained MoE Upcycling

Jun 15, 2026

This work addresses the challenge in data-constrained supervised fine-tuning of Mixture-of-Experts (MoE) models, where existing adaptation methods often induce expert homogenization or excessively perturb pretrained parameters, failing to balance architectural reuse with diversity. To overcome this, we propose SVD-Partitioned Residual Initialization (SPRI), which— for the first time— decomposes the pretrained feed-forward network (FFN) weights via singular value decomposition (SVD) and allocates the resulting residual components across distinct experts. This approach preserves the spectral structure of the original weights while introducing controlled diversity, further enhanced by a two-stage training strategy to improve stability. Evaluated on 15 English-to-many translation directions in CoVoST2, SPRI achieves average gains of 2.58 BLEU and 3.32 COMET points over fully fine-tuned dense models, and outperforms the previous best MoE method by 3.39 BLEU and 4.34 COMET points.

0 citationsRead paper