Selected work

Representative Papers

TCPO: Turn-Level Credit Policy Optimization

Aug 03, 2026

This work addresses the credit assignment problem in multi-turn reasoning, where episode-level verifier scores inadequately reflect individual action contributions to final outcomes. To resolve this, the authors propose TCPO, a method that transforms coarse episode-level rewards into fine-grained credit signals through a reference-state comparison mechanism, augmented with retrospective credit, hindsight-delayed credit, and selective counterfactual estimation. By integrating episode-level advantage construction with guidance from multiple verifier types, TCPO enables precise action-value evaluation. Empirical results demonstrate substantial improvements over existing baselines across mathematical reasoning, code generation, and AppWorld tasks, reducing the number of turns required for success and achieving state-of-the-art or tied-best Pass@8 performance on Qwen3-4B and DeepSeek-R1-Distill-Llama-8B.

0 citationsRead paper

LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

Aug 03, 2026

This work addresses the challenges of sparse rewards, high compilation latency, and signal dilution across multi-round feedback in reinforcement learning for low-level system programming tasks such as CUDA kernel generation. The authors propose LEAP, a novel framework that integrates difficulty-conditioned pruning (DCP) with a critic-free multi-round reinforcement learning paradigm. By dynamically pruning tasks to focus on high-value, complex samples and employing a ranking-based pairwise tournament reward mechanism, LEAP enables adaptive optimization without manual hyperparameter tuning. The framework combines Group Relative Policy Optimization (GRPO) with hardware-aware multi-round environment feedback, maintaining strong initial-generation performance while significantly improving robustness and convergence speed during iterative refinement—offering an efficient and practical reinforcement learning solution for low-level code generation.

0 citationsRead paper

Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold

Jul 30, 2026

This work addresses the limited reasoning coverage of single reinforcement learning teacher models, which constrains their ability to comprehensively guide student models. To overcome this, the authors propose an “Expand-and-Compress” framework: first, a set of complementary teachers is constructed through iterative training, leveraging Residual Group Relative Policy Optimization (RGRPO) to broaden the solution space while preserving expert behaviors via Consensus-Residual Decomposition; subsequently, knowledge from this ensemble is efficiently distilled into a single student model using reliability-weighted Teacher-Union On-policy Distillation (TU-OPD). Experimental results demonstrate that the resulting Qwen3-1.7B student model outperforms the strongest individual teacher by 2.0%, 8.3%, and 6.9% on mathematical reasoning, code generation, and instruction-following tasks, respectively, while retaining the inference efficiency of a single model.

0 citationsRead paper

MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPU

Jun 03, 2026

This work addresses the challenges faced by current large language models in generating efficient and correct native GPU kernel code, as well as the instability and sparse reward issues inherent in execution-based reinforcement learning. We propose MusaCoder, a full-stack training framework tailored for CUDA and MUSA backends, which enhances code generation quality through progressive kernel synthesis, diversity-preserving rejection fine-tuning, and execution-feedback-driven reinforcement learning. Our approach introduces three key innovations: PrimeEcho, a multi-round reward mechanism; Buffered Dynamic Retry, a hard-example recovery strategy; and MirrorPop, an off-policy sequence filtering method—collectively improving training stability. Evaluated on KernelBench and a MUSA porting benchmark, our 9B model matches leading closed-source systems, while the 27B variant establishes a new state of the art, significantly outperforming both open-source and commercial baselines in correctness and empirical speedup.

0 citationsRead paper

LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning

May 08, 2026

This work addresses the performance bottlenecks of lightweight on-device vision-language GUI agents caused by limited model capacity, as well as issues of overfitting, catastrophic forgetting, and policy rigidity induced by supervised fine-tuning. To overcome these challenges, the paper proposes a novel training paradigm that eliminates the need for supervised fine-tuning. It introduces, for the first time, a systematic application of general knowledge distillation to GUI agents, integrating guided online policy distillation, a dynamic retrieval mechanism, and an automated multi-solution trajectory generation pipeline. Furthermore, a multi-solution dual-level GRPO reinforcement learning framework is designed to jointly optimize subtask planning and action execution. Experiments demonstrate that the proposed method achieves state-of-the-art performance among lightweight models across all benchmarks, matching or surpassing significantly larger models and substantially unlocking the potential of 2B/3B-scale agents beyond conventional imitation learning.

0 citationsRead paper

Recent publications

Latest Papers

TCPO: Turn-Level Credit Policy Optimization

Aug 03, 2026

This work addresses the credit assignment problem in multi-turn reasoning, where episode-level verifier scores inadequately reflect individual action contributions to final outcomes. To resolve this, the authors propose TCPO, a method that transforms coarse episode-level rewards into fine-grained credit signals through a reference-state comparison mechanism, augmented with retrospective credit, hindsight-delayed credit, and selective counterfactual estimation. By integrating episode-level advantage construction with guidance from multiple verifier types, TCPO enables precise action-value evaluation. Empirical results demonstrate substantial improvements over existing baselines across mathematical reasoning, code generation, and AppWorld tasks, reducing the number of turns required for success and achieving state-of-the-art or tied-best Pass@8 performance on Qwen3-4B and DeepSeek-R1-Distill-Llama-8B.

0 citationsRead paper

LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation

Aug 03, 2026

This work addresses the challenges of sparse rewards, high compilation latency, and signal dilution across multi-round feedback in reinforcement learning for low-level system programming tasks such as CUDA kernel generation. The authors propose LEAP, a novel framework that integrates difficulty-conditioned pruning (DCP) with a critic-free multi-round reinforcement learning paradigm. By dynamically pruning tasks to focus on high-value, complex samples and employing a ranking-based pairwise tournament reward mechanism, LEAP enables adaptive optimization without manual hyperparameter tuning. The framework combines Group Relative Policy Optimization (GRPO) with hardware-aware multi-round environment feedback, maintaining strong initial-generation performance while significantly improving robustness and convergence speed during iterative refinement—offering an efficient and practical reinforcement learning solution for low-level code generation.

0 citationsRead paper

Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold

Jul 30, 2026

This work addresses the limited reasoning coverage of single reinforcement learning teacher models, which constrains their ability to comprehensively guide student models. To overcome this, the authors propose an “Expand-and-Compress” framework: first, a set of complementary teachers is constructed through iterative training, leveraging Residual Group Relative Policy Optimization (RGRPO) to broaden the solution space while preserving expert behaviors via Consensus-Residual Decomposition; subsequently, knowledge from this ensemble is efficiently distilled into a single student model using reliability-weighted Teacher-Union On-policy Distillation (TU-OPD). Experimental results demonstrate that the resulting Qwen3-1.7B student model outperforms the strongest individual teacher by 2.0%, 8.3%, and 6.9% on mathematical reasoning, code generation, and instruction-following tasks, respectively, while retaining the inference efficiency of a single model.

0 citationsRead paper

MusaCoder: Native GPU Kernel Generation with Full-Stack Training on Moore Threads GPU

Jun 03, 2026

This work addresses the challenges faced by current large language models in generating efficient and correct native GPU kernel code, as well as the instability and sparse reward issues inherent in execution-based reinforcement learning. We propose MusaCoder, a full-stack training framework tailored for CUDA and MUSA backends, which enhances code generation quality through progressive kernel synthesis, diversity-preserving rejection fine-tuning, and execution-feedback-driven reinforcement learning. Our approach introduces three key innovations: PrimeEcho, a multi-round reward mechanism; Buffered Dynamic Retry, a hard-example recovery strategy; and MirrorPop, an off-policy sequence filtering method—collectively improving training stability. Evaluated on KernelBench and a MUSA porting benchmark, our 9B model matches leading closed-source systems, while the 27B variant establishes a new state of the art, significantly outperforming both open-source and commercial baselines in correctness and empirical speedup.

0 citationsRead paper

LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning

May 08, 2026

This work addresses the performance bottlenecks of lightweight on-device vision-language GUI agents caused by limited model capacity, as well as issues of overfitting, catastrophic forgetting, and policy rigidity induced by supervised fine-tuning. To overcome these challenges, the paper proposes a novel training paradigm that eliminates the need for supervised fine-tuning. It introduces, for the first time, a systematic application of general knowledge distillation to GUI agents, integrating guided online policy distillation, a dynamic retrieval mechanism, and an automated multi-solution trajectory generation pipeline. Furthermore, a multi-solution dual-level GRPO reinforcement learning framework is designed to jointly optimize subtask planning and action execution. Experiments demonstrate that the proposed method achieves state-of-the-art performance among lightweight models across all benchmarks, matching or surpassing significantly larger models and substantially unlocking the potential of 2B/3B-scale agents beyond conventional imitation learning.

0 citationsRead paper