Selected work

Representative Papers

Efficient Test-Time Optimization for Multi-Agent Proof Autoformalization

Jul 13, 2026

This work addresses the challenge of automatically translating multi-step mathematical proofs from natural language into formal languages by proposing ToMap, a multi-agent framework. ToMap employs a decomposer-formalizer-prover pipeline and identifies the decomposer as the critical bottleneck, innovatively concentrating test-time computation on its iterative refinement. By integrating formal verification feedback, semantic proof scoring, and a GEPA-inspired Pareto-front guidance mechanism, the framework jointly enhances syntactic correctness and semantic fidelity. Evaluated on ProofFlowBench, ToMap outperforms the previous state-of-the-art method by 19.0%, with most performance gains achieved within just a few iterations while simultaneously reducing overall test-time computational overhead.

0 citationsRead paper

Offline Multi-agent Continual Cooperation via Skill Partition and Reuse

Jun 24, 2026

This work addresses catastrophic forgetting and loss of plasticity in multi-agent offline continuous control, which arise from fixed skill repositories and distributional shifts. To tackle these challenges, the paper proposes COMAD, a novel framework that integrates density estimation–driven identification of skill reusability with a multi-headed policy architecture. COMAD employs an autoencoder to extract coordinated skills from heterogeneous behavioral data and formulates a skill-augmented policy objective that enables dynamic expansion of the skill library. Theoretical analysis provides guarantees for the optimality of skill discovery. Empirical results demonstrate that COMAD significantly outperforms existing methods across multiple multi-agent reinforcement learning benchmarks, effectively mitigating task interference and achieving strong forward and backward transfer capabilities.

0 citationsRead paper

Model-Based Proactive Cost Generation for Learning Safe Policies Offline with Limited Violation Data

May 02, 2026

This work addresses the challenge of offline reinforcement learning in high-stakes environments, where the scarcity of unsafe samples hinders the identification of latent constraint-violating states and often leads to deployment failures. To overcome this limitation, the authors propose the PROCO framework, which first learns a dynamics model from offline data and then leverages a large language model (LLM) to inject safety priors expressed in natural language. These priors inform a conservative cost function, enabling the generation of counterfactual unsafe trajectories through model-based rollouts. This approach proactively delineates the feasible region and facilitates safe policy learning without relying on real-world violations. PROCO is the first method to integrate semantic safety knowledge from LLMs into offline safe reinforcement learning, achieving risk anticipation and synthetic unsafe sample generation under near-zero violation conditions. Experiments on the Safety-Gymnasium benchmark demonstrate that PROCO substantially reduces constraint violations and seamlessly enhances diverse offline safe RL algorithms, outperforming both their original variants and behavior cloning baselines.

0 citationsRead paper

ReinVBC: A Model-based Reinforcement Learning Approach to Vehicle Braking Controller

Apr 06, 2026

This work proposes an offline model-based reinforcement learning approach that integrates engineering priors to address the inefficiency of traditional vehicle braking controllers, which rely heavily on manual calibration. By leveraging data-driven techniques, the method constructs a high-fidelity vehicle dynamics model and subsequently optimizes braking strategies without requiring online interaction with the environment. The resulting controller achieves braking performance comparable to production-grade anti-lock braking systems (ABS) in real-world braking tasks, demonstrating significant potential as a viable alternative to current industry-standard systems.

0 citationsRead paper

NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios

Mar 25, 2025

Existing offline reinforcement learning (Offline RL) benchmarks severely underestimate critical challenges in real-world deployment: extremely conservative data distributions, significant action delays, uncontrollable environmental disturbances, and globally defined safety constraints that are difficult to model and verify. To address this gap, we propose NeoRL-2—the first systematic, extended Offline RL benchmark explicitly designed to capture these four industrial-grade challenges. Built upon high-fidelity physics simulation and controllable perturbation generation, NeoRL-2 comprises seven realistic tasks, each accompanied by an evaluation-capable simulator supporting behavioral policy comparison, formal safety verification, and generalization assessment. Empirical evaluation reveals that mainstream Offline RL algorithms consistently fail to outperform the original behavior policies, exposing fundamental limitations in robustness and safety compliance. NeoRL-2 establishes a standardized, reproducible evaluation platform and benchmark baseline to advance the reliable, industrial-scale deployment of Offline RL.

0 citationsRead paper

Recent publications

Latest Papers

Efficient Test-Time Optimization for Multi-Agent Proof Autoformalization

Jul 13, 2026

This work addresses the challenge of automatically translating multi-step mathematical proofs from natural language into formal languages by proposing ToMap, a multi-agent framework. ToMap employs a decomposer-formalizer-prover pipeline and identifies the decomposer as the critical bottleneck, innovatively concentrating test-time computation on its iterative refinement. By integrating formal verification feedback, semantic proof scoring, and a GEPA-inspired Pareto-front guidance mechanism, the framework jointly enhances syntactic correctness and semantic fidelity. Evaluated on ProofFlowBench, ToMap outperforms the previous state-of-the-art method by 19.0%, with most performance gains achieved within just a few iterations while simultaneously reducing overall test-time computational overhead.

0 citationsRead paper

Offline Multi-agent Continual Cooperation via Skill Partition and Reuse

Jun 24, 2026

This work addresses catastrophic forgetting and loss of plasticity in multi-agent offline continuous control, which arise from fixed skill repositories and distributional shifts. To tackle these challenges, the paper proposes COMAD, a novel framework that integrates density estimation–driven identification of skill reusability with a multi-headed policy architecture. COMAD employs an autoencoder to extract coordinated skills from heterogeneous behavioral data and formulates a skill-augmented policy objective that enables dynamic expansion of the skill library. Theoretical analysis provides guarantees for the optimality of skill discovery. Empirical results demonstrate that COMAD significantly outperforms existing methods across multiple multi-agent reinforcement learning benchmarks, effectively mitigating task interference and achieving strong forward and backward transfer capabilities.

0 citationsRead paper

Model-Based Proactive Cost Generation for Learning Safe Policies Offline with Limited Violation Data

May 02, 2026

This work addresses the challenge of offline reinforcement learning in high-stakes environments, where the scarcity of unsafe samples hinders the identification of latent constraint-violating states and often leads to deployment failures. To overcome this limitation, the authors propose the PROCO framework, which first learns a dynamics model from offline data and then leverages a large language model (LLM) to inject safety priors expressed in natural language. These priors inform a conservative cost function, enabling the generation of counterfactual unsafe trajectories through model-based rollouts. This approach proactively delineates the feasible region and facilitates safe policy learning without relying on real-world violations. PROCO is the first method to integrate semantic safety knowledge from LLMs into offline safe reinforcement learning, achieving risk anticipation and synthetic unsafe sample generation under near-zero violation conditions. Experiments on the Safety-Gymnasium benchmark demonstrate that PROCO substantially reduces constraint violations and seamlessly enhances diverse offline safe RL algorithms, outperforming both their original variants and behavior cloning baselines.

0 citationsRead paper

ReinVBC: A Model-based Reinforcement Learning Approach to Vehicle Braking Controller

Apr 06, 2026

This work proposes an offline model-based reinforcement learning approach that integrates engineering priors to address the inefficiency of traditional vehicle braking controllers, which rely heavily on manual calibration. By leveraging data-driven techniques, the method constructs a high-fidelity vehicle dynamics model and subsequently optimizes braking strategies without requiring online interaction with the environment. The resulting controller achieves braking performance comparable to production-grade anti-lock braking systems (ABS) in real-world braking tasks, demonstrating significant potential as a viable alternative to current industry-standard systems.

0 citationsRead paper

NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios

Mar 25, 2025

Existing offline reinforcement learning (Offline RL) benchmarks severely underestimate critical challenges in real-world deployment: extremely conservative data distributions, significant action delays, uncontrollable environmental disturbances, and globally defined safety constraints that are difficult to model and verify. To address this gap, we propose NeoRL-2—the first systematic, extended Offline RL benchmark explicitly designed to capture these four industrial-grade challenges. Built upon high-fidelity physics simulation and controllable perturbation generation, NeoRL-2 comprises seven realistic tasks, each accompanied by an evaluation-capable simulator supporting behavioral policy comparison, formal safety verification, and generalization assessment. Empirical evaluation reveals that mainstream Offline RL algorithms consistently fail to outperform the original behavior policies, exposing fundamental limitations in robustness and safety compliance. NeoRL-2 establishes a standardized, reproducible evaluation platform and benchmark baseline to advance the reliable, industrial-scale deployment of Offline RL.

0 citationsRead paper