Selected work

Representative Papers

StepOPSD: Step-Aware Online Preference Distillation for Agent Reinforcement Learning

May 26, 2026

This work addresses the credit assignment mismatch between sparse trajectory rewards and critical local actions in multi-turn agent reinforcement learning. It proposes StepOPSD, a novel framework that refines credit assignment to the level of individual agent steps for the first time. StepOPSD guides GRPO policy updates through step-level trajectory decomposition, hindsight-augmented teacher context re-scoring, sign-preserving advantage shaping, and normalized credit budgeting. Additionally, it introduces a dual-parameter control mechanism—α_clip and λ_mix—to modulate learning dynamics. The method achieves state-of-the-art performance on ALFWorld (e.g., 79.1% on Heat, 95.0% on PickTwo) and Search-QA (61.6% on TriviaQA). Empirical analysis further reveals that α_clip stabilizes local trust regions, while λ_mix exhibits task-dependent tuning characteristics.

0 citationsRead paper

Scalable Environments Drive Generalizable Agents

May 18, 2026

Current agents exhibit limited generalization when confronted with out-of-distribution environmental changes, such as shifts in interaction rules, dynamics, or observation feedback. This work proposes “environment expansion”—enhancing cross-environment generalization by broadening the distribution of executable rule sets an agent interacts with, rather than merely increasing the number of trajectories or tasks. We formally distinguish trajectory expansion, task expansion, and environment expansion, establishing a unified taxonomy and highlighting that distributional expansion at the environment level is essential for robust, general-purpose agents. Scalable environments are constructed via two paradigms: procedural generators and generative world models, integrated with state-aware learning mechanisms to enable cross-environment adaptation. This study provides a theoretical framework and technical pathway toward measurable and controllable general agents, significantly improving their adaptability and robustness in unseen environments.

0 citationsRead paper

Harnessing Agentic Evolution

May 13, 2026

Existing agent evolution methods often suffer from rigidity that limits adaptability or drift from objectives over long-term evolution, while also struggling to systematically leverage historical evolutionary evidence. This work proposes AEvo, a novel framework that, for the first time, formulates agent evolution as an interactive environment. AEvo introduces a meta-agent that dynamically edits subsequent evolution mechanisms based on contextual evolutionary states, rather than directly generating candidate solutions, thereby unifying the control of both procedural and agent-based evolutionary processes. By incorporating evolutionary trajectory tracking and context management, the framework enables systematic reuse of historical evidence. Experiments demonstrate that AEvo outperforms five baselines across multiple reasoning and agent benchmarks, achieving a 26% improvement over the strongest baseline, and attains state-of-the-art performance under equivalent iteration budgets in three open-ended optimization tasks.

0 citationsRead paper

Co-Evolution of Policy and Internal Reward for Language Agents

Apr 03, 2026

This work addresses the challenge that large language model agents often struggle with sparse and delayed environmental rewards in long-horizon tasks, leading to insufficient learning signals. To overcome this limitation, the authors propose a Self-Guide mechanism that generates internal self-guidance signals during inference to inform action selection and, during training, transforms these signals into dense internal rewards. These rewards are jointly optimized with the policy via a novel Guided Reinforcement Policy Optimization (GRPO) algorithm. This approach represents the first method to enable co-evolution of internal rewards and policy in language agents, unifying inference-time guidance with training-time supervision. Experimental results demonstrate that incorporating self-guidance during inference alone yields substantial performance gains; when combined with GRPO-based joint training, the method outperforms pure environment-reward baselines by an average of 8% across three benchmarks.

0 citationsRead paper

The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook

Apr 02, 2026

This work addresses the limitations of explicit symbolic spaces in language models—namely linguistic redundancy, discretization bottlenecks, sequential inefficiency, and semantic loss—which constrain computational efficiency and expressive capacity. The study systematically reviews advances in latent space research and introduces, for the first time, a five-dimensional analytical framework tailored to language models: foundations, evolution, mechanisms, capabilities, and outlook. This unified framework integrates architecture, representation, computation, and optimization, while linking technical pathways to higher-order abilities such as reasoning, memory, and embodiment. By synthesizing and categorizing cutting-edge research, the paper elucidates the pivotal role of latent spaces in enhancing model efficiency and capability, and clearly identifies key challenges and promising directions for future work.

0 citationsRead paper

Recent publications

Latest Papers

StepOPSD: Step-Aware Online Preference Distillation for Agent Reinforcement Learning

May 26, 2026

This work addresses the credit assignment mismatch between sparse trajectory rewards and critical local actions in multi-turn agent reinforcement learning. It proposes StepOPSD, a novel framework that refines credit assignment to the level of individual agent steps for the first time. StepOPSD guides GRPO policy updates through step-level trajectory decomposition, hindsight-augmented teacher context re-scoring, sign-preserving advantage shaping, and normalized credit budgeting. Additionally, it introduces a dual-parameter control mechanism—α_clip and λ_mix—to modulate learning dynamics. The method achieves state-of-the-art performance on ALFWorld (e.g., 79.1% on Heat, 95.0% on PickTwo) and Search-QA (61.6% on TriviaQA). Empirical analysis further reveals that α_clip stabilizes local trust regions, while λ_mix exhibits task-dependent tuning characteristics.

0 citationsRead paper

Scalable Environments Drive Generalizable Agents

May 18, 2026

Current agents exhibit limited generalization when confronted with out-of-distribution environmental changes, such as shifts in interaction rules, dynamics, or observation feedback. This work proposes “environment expansion”—enhancing cross-environment generalization by broadening the distribution of executable rule sets an agent interacts with, rather than merely increasing the number of trajectories or tasks. We formally distinguish trajectory expansion, task expansion, and environment expansion, establishing a unified taxonomy and highlighting that distributional expansion at the environment level is essential for robust, general-purpose agents. Scalable environments are constructed via two paradigms: procedural generators and generative world models, integrated with state-aware learning mechanisms to enable cross-environment adaptation. This study provides a theoretical framework and technical pathway toward measurable and controllable general agents, significantly improving their adaptability and robustness in unseen environments.

0 citationsRead paper

Harnessing Agentic Evolution

May 13, 2026

Existing agent evolution methods often suffer from rigidity that limits adaptability or drift from objectives over long-term evolution, while also struggling to systematically leverage historical evolutionary evidence. This work proposes AEvo, a novel framework that, for the first time, formulates agent evolution as an interactive environment. AEvo introduces a meta-agent that dynamically edits subsequent evolution mechanisms based on contextual evolutionary states, rather than directly generating candidate solutions, thereby unifying the control of both procedural and agent-based evolutionary processes. By incorporating evolutionary trajectory tracking and context management, the framework enables systematic reuse of historical evidence. Experiments demonstrate that AEvo outperforms five baselines across multiple reasoning and agent benchmarks, achieving a 26% improvement over the strongest baseline, and attains state-of-the-art performance under equivalent iteration budgets in three open-ended optimization tasks.

0 citationsRead paper

Co-Evolution of Policy and Internal Reward for Language Agents

Apr 03, 2026

This work addresses the challenge that large language model agents often struggle with sparse and delayed environmental rewards in long-horizon tasks, leading to insufficient learning signals. To overcome this limitation, the authors propose a Self-Guide mechanism that generates internal self-guidance signals during inference to inform action selection and, during training, transforms these signals into dense internal rewards. These rewards are jointly optimized with the policy via a novel Guided Reinforcement Policy Optimization (GRPO) algorithm. This approach represents the first method to enable co-evolution of internal rewards and policy in language agents, unifying inference-time guidance with training-time supervision. Experimental results demonstrate that incorporating self-guidance during inference alone yields substantial performance gains; when combined with GRPO-based joint training, the method outperforms pure environment-reward baselines by an average of 8% across three benchmarks.

0 citationsRead paper

The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook

Apr 02, 2026

This work addresses the limitations of explicit symbolic spaces in language models—namely linguistic redundancy, discretization bottlenecks, sequential inefficiency, and semantic loss—which constrain computational efficiency and expressive capacity. The study systematically reviews advances in latent space research and introduces, for the first time, a five-dimensional analytical framework tailored to language models: foundations, evolution, mechanisms, capabilities, and outlook. This unified framework integrates architecture, representation, computation, and optimization, while linking technical pathways to higher-order abilities such as reasoning, memory, and embodiment. By synthesizing and categorizing cutting-edge research, the paper elucidates the pivotal role of latent spaces in enhancing model efficiency and capability, and clearly identifies key challenges and promising directions for future work.

0 citationsRead paper