Selected work

Representative Papers

rePIRL: Learn PRM with Inverse RL for LLM Reasoning

Feb 08, 2026

Existing process reward model (PRM) approaches either rely on strong expert assumptions or are prone to entropy collapse, limiting their generalization. This work proposes rePIRL, a novel framework that introduces inverse reinforcement learning into PRM learning for large language model (LLM) reasoning. By alternately optimizing the policy and the PRM, rePIRL establishes a dual-learning mechanism that unifies online and offline training under weak expert assumptions and effectively mitigates entropy collapse. Experiments demonstrate that rePIRL significantly outperforms existing methods on mathematical and code reasoning tasks. The learned PRMs prove effective for test-time training, test-time scaling, and early signal generation on challenging problems. Ablation studies further confirm the contribution of each component in the proposed design.

0 citationsRead paper

A Fully First-Order Layer for Differentiable Optimization

Dec 02, 2025

Traditional differentiable optimization layers rely on implicit differentiation, requiring the solution of Hessian-involved linear systems—entailing prohibitive computational and memory costs. This paper proposes FFOLayer, a fully first-order differentiable optimization layer: it reformulates embedded optimization as a bilevel problem and leverages active-set identification coupled with a Lagrangian hypergradient mechanism to achieve finite-time, non-asymptotic hypergradient approximation using only first-order information. Theoretically, FFOLayer attains a convergence rate matching that of optimal nonsmooth nonconvex methods, with overall complexity $ ilde{mathcal{O}}(delta^{-1}epsilon^{-3})$. Crucially, it eliminates Hessian computation and storage, drastically reducing overhead. Furthermore, we release an open-source, plug-and-play Python library supporting seamless integration with mainstream deep learning frameworks and optimization solvers.

0 citationsRead paper

RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage

Feb 13, 2025

This work addresses the dual security threats—prompt injection attacks and privacy leakage—in tool-based agent systems (TBAS), where language model agents are inherently vulnerable. We propose an automated, user-intervention-free defense framework. First, we adapt information-flow control (IFC) to the TBAS setting—a novel application in this domain. Second, we design a dual-dependency screener leveraging large language model (LLM) discrimination and attention saliency to dynamically assess tool invocation permissions and perform sensitivity analysis. Evaluated on the AgentDojo benchmark, our method achieves 100% success in blocking targeted attacks while incurring only a 2% drop in task utility; it further attains near-oracular accuracy in detecting both explicit and implicit privacy leaks. Our core contribution lies in the first integration of IFC with LLM-based self-assessment mechanisms, striking a practical balance between security assurance and operational efficiency, and substantially reducing reliance on manual verification.

0 citationsRead paper

Recent publications

Latest Papers

rePIRL: Learn PRM with Inverse RL for LLM Reasoning

Feb 08, 2026

Existing process reward model (PRM) approaches either rely on strong expert assumptions or are prone to entropy collapse, limiting their generalization. This work proposes rePIRL, a novel framework that introduces inverse reinforcement learning into PRM learning for large language model (LLM) reasoning. By alternately optimizing the policy and the PRM, rePIRL establishes a dual-learning mechanism that unifies online and offline training under weak expert assumptions and effectively mitigates entropy collapse. Experiments demonstrate that rePIRL significantly outperforms existing methods on mathematical and code reasoning tasks. The learned PRMs prove effective for test-time training, test-time scaling, and early signal generation on challenging problems. Ablation studies further confirm the contribution of each component in the proposed design.

0 citationsRead paper

A Fully First-Order Layer for Differentiable Optimization

Dec 02, 2025

Traditional differentiable optimization layers rely on implicit differentiation, requiring the solution of Hessian-involved linear systems—entailing prohibitive computational and memory costs. This paper proposes FFOLayer, a fully first-order differentiable optimization layer: it reformulates embedded optimization as a bilevel problem and leverages active-set identification coupled with a Lagrangian hypergradient mechanism to achieve finite-time, non-asymptotic hypergradient approximation using only first-order information. Theoretically, FFOLayer attains a convergence rate matching that of optimal nonsmooth nonconvex methods, with overall complexity $ ilde{mathcal{O}}(delta^{-1}epsilon^{-3})$. Crucially, it eliminates Hessian computation and storage, drastically reducing overhead. Furthermore, we release an open-source, plug-and-play Python library supporting seamless integration with mainstream deep learning frameworks and optimization solvers.

0 citationsRead paper

RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage

Feb 13, 2025

This work addresses the dual security threats—prompt injection attacks and privacy leakage—in tool-based agent systems (TBAS), where language model agents are inherently vulnerable. We propose an automated, user-intervention-free defense framework. First, we adapt information-flow control (IFC) to the TBAS setting—a novel application in this domain. Second, we design a dual-dependency screener leveraging large language model (LLM) discrimination and attention saliency to dynamically assess tool invocation permissions and perform sensitivity analysis. Evaluated on the AgentDojo benchmark, our method achieves 100% success in blocking targeted attacks while incurring only a 2% drop in task utility; it further attains near-oracular accuracy in detecting both explicit and implicit privacy leaks. Our core contribution lies in the first integration of IFC with LLM-based self-assessment mechanisms, striking a practical balance between security assurance and operational efficiency, and substantially reducing reliance on manual verification.

0 citationsRead paper