Selected work

Representative Papers

Implicit Contrastive Representation Learning with Guided Stop-gradient

Mar 12, 2025Neural Information Processing Systems

Self-supervised learning faces two key challenges: Siamese networks suffer from representational collapse, while contrastive learning relies on negative samples, leading to poor robustness under small batch sizes. To address these issues, this paper proposes a negative-sample-free implicit contrastive learning paradigm. Its core innovation is a guided stop-gradient mechanism that dynamically blocks gradients between symmetric positive sample pairs, thereby implicitly encoding contrastive signals without requiring negative samples, prediction heads, or asymmetric encoders. The method is fully compatible with SimSiam and BYOL frameworks, needing only standard momentum updates and symmetric loss functions. Extensive experiments demonstrate significant performance gains on ImageNet, robust training with extremely small batch sizes (e.g., 8), and superior training stability and generalization compared to existing negative-sample-free approaches.

1 citationsRead paper

AETDICE: Unified Framework and Offline Optimization for Nonlinear Multi-Objective RL

Jun 30, 2026

In nonlinear multi-objective reinforcement learning, the scalarized expected return (SER) and expected scalarized return (ESR) paradigms have long remained disjoint due to fundamental differences in their optimization levels and policy characteristics, hindering unified modeling of complex preferences such as risk aversion or fairness. This work proposes the Aggregation-Expectation-Transformation (AET) framework, which reconciles SER and ESR through a tripartite decomposition and introduces AETDICE—the first nonlinear multi-objective offline reinforcement learning algorithm capable of training from static datasets. By integrating DICE-style density ratio estimation within an augmented state space, AETDICE enables effective off-policy optimization and overcomes the longstanding trade-off limitations between SER and ESR in existing approaches.

0 citationsRead paper

Active Query Synthesis for Preference Learning

May 25, 2026

This work addresses the challenge of user preference learning, which typically relies on costly annotated data, while existing active learning approaches suffer from high computational overhead and fail to account for varying reliability in user feedback. The authors propose Info-Synth, a novel framework that uniquely integrates confidence-aware response modeling with active query synthesis in continuous space. By maximizing mutual information, Info-Synth generates highly informative preference queries and introduces two strategies—Pair M-dist and Pair Opt-dist—to effectively handle ambiguous comparisons. The method demonstrates substantially improved learning efficiency, outperforming baseline approaches across diverse tasks including synthetic preference learning, text summarization, and robot controller tuning. Furthermore, it naturally extends to practical scenarios with limited query pools.

0 citationsRead paper

Score-Based One-step MeanFlow Policy Optimization

May 22, 2026

This work addresses the high computational cost and reliance on target-distribution samples that plague existing multi-step denoising strategies based on MeanFlow in online reinforcement learning. To overcome these limitations, we propose the Single-step MeanFlow policy Optimization Method (SOM), which, for the first time, constructs a target velocity field in a fully online setting without requiring any target samples. SOM estimates the score function via the Q-function and leverages a probability flow ordinary differential equation (ODE) to generate actions, enabling policy sampling with only a single neural network forward pass. By reducing multi-step generation to a single step, the method achieves state-of-the-art performance on motion control tasks while substantially decreasing both training and inference time.

0 citationsRead paper

Direct Soft-Policy Sampling via Langevin Dynamics

Feb 08, 2026

This work proposes Noise-Conditioned Langevin Q-Learning (NC-LQL), a novel soft policy optimization framework for reinforcement learning that circumvents the need for explicit policy parameterization. Instead of representing a policy directly, NC-LQL samples actions from the Boltzmann distribution induced by the Q-function via Langevin dynamics. To enhance exploration and stabilize learning, the method incorporates multi-scale noise perturbations and a noise-conditioned Q-function, which together shape a progressively smoothed value landscape—enabling efficient transitions from global exploration to local refinement. Evaluated on the OpenAI Gym MuJoCo benchmark suite, NC-LQL achieves performance comparable to state-of-the-art diffusion-based approaches, offering a concise and effective alternative for online reinforcement learning without the limitations of policy expressivity or entropy estimation inherent in conventional soft policy methods.

0 citationsRead paper

Recent publications

Latest Papers

AETDICE: Unified Framework and Offline Optimization for Nonlinear Multi-Objective RL

Jun 30, 2026

In nonlinear multi-objective reinforcement learning, the scalarized expected return (SER) and expected scalarized return (ESR) paradigms have long remained disjoint due to fundamental differences in their optimization levels and policy characteristics, hindering unified modeling of complex preferences such as risk aversion or fairness. This work proposes the Aggregation-Expectation-Transformation (AET) framework, which reconciles SER and ESR through a tripartite decomposition and introduces AETDICE—the first nonlinear multi-objective offline reinforcement learning algorithm capable of training from static datasets. By integrating DICE-style density ratio estimation within an augmented state space, AETDICE enables effective off-policy optimization and overcomes the longstanding trade-off limitations between SER and ESR in existing approaches.

0 citationsRead paper

Active Query Synthesis for Preference Learning

May 25, 2026

This work addresses the challenge of user preference learning, which typically relies on costly annotated data, while existing active learning approaches suffer from high computational overhead and fail to account for varying reliability in user feedback. The authors propose Info-Synth, a novel framework that uniquely integrates confidence-aware response modeling with active query synthesis in continuous space. By maximizing mutual information, Info-Synth generates highly informative preference queries and introduces two strategies—Pair M-dist and Pair Opt-dist—to effectively handle ambiguous comparisons. The method demonstrates substantially improved learning efficiency, outperforming baseline approaches across diverse tasks including synthetic preference learning, text summarization, and robot controller tuning. Furthermore, it naturally extends to practical scenarios with limited query pools.

0 citationsRead paper

Score-Based One-step MeanFlow Policy Optimization

May 22, 2026

This work addresses the high computational cost and reliance on target-distribution samples that plague existing multi-step denoising strategies based on MeanFlow in online reinforcement learning. To overcome these limitations, we propose the Single-step MeanFlow policy Optimization Method (SOM), which, for the first time, constructs a target velocity field in a fully online setting without requiring any target samples. SOM estimates the score function via the Q-function and leverages a probability flow ordinary differential equation (ODE) to generate actions, enabling policy sampling with only a single neural network forward pass. By reducing multi-step generation to a single step, the method achieves state-of-the-art performance on motion control tasks while substantially decreasing both training and inference time.

0 citationsRead paper

Direct Soft-Policy Sampling via Langevin Dynamics

Feb 08, 2026

This work proposes Noise-Conditioned Langevin Q-Learning (NC-LQL), a novel soft policy optimization framework for reinforcement learning that circumvents the need for explicit policy parameterization. Instead of representing a policy directly, NC-LQL samples actions from the Boltzmann distribution induced by the Q-function via Langevin dynamics. To enhance exploration and stabilize learning, the method incorporates multi-scale noise perturbations and a noise-conditioned Q-function, which together shape a progressively smoothed value landscape—enabling efficient transitions from global exploration to local refinement. Evaluated on the OpenAI Gym MuJoCo benchmark suite, NC-LQL achieves performance comparable to state-of-the-art diffusion-based approaches, offering a concise and effective alternative for online reinforcement learning without the limitations of policy expressivity or entropy estimation inherent in conventional soft policy methods.

0 citationsRead paper

TABX: A High-Throughput Sandbox Battle Simulator for Multi-Agent Reinforcement Learning

Feb 02, 2026

This work proposes the first JAX-based multi-agent reinforcement learning (MARL) sandbox framework that is highly parallel, scalable, and extensively customizable, addressing the lack of modularity in existing MARL benchmark environments. The framework enables fine-grained control and rapid configuration of environmental parameters, facilitating flexible construction and evaluation of user-defined scenarios. By leveraging GPU hardware acceleration, it supports large-scale parallel simulation, substantially improving experimental throughput and flexibility while significantly reducing computational overhead. This provides an efficient and reconfigurable foundation for advancing MARL algorithm research in complex, structured domains.

0 citationsRead paper