Selected work

Representative Papers

Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMs

Nov 04, 2025

This work addresses the challenge of efficient learning and generalization for agents operating under stringent trajectory constraints—such as resource limitations or safety-critical requirements—during deployment. To this end, we propose a progressive curriculum learning framework. Methodologically, we introduce curriculum learning to trajectory-constrained reinforcement learning for the first time, employing a self-paced strategy that dynamically tightens constraints from loose to strict. We further integrate binary-tree MDP modeling, a multi-task navigation architecture, and LLM-based chain-of-thought (CoT) compression and inference acceleration at the token level. Experiments demonstrate substantial improvements in training efficiency and policy robustness, validate strong generalization across diverse constraint settings, and achieve significant CoT compression and inference speedup in LLM-driven reasoning.

0 citationsRead paper

Inference-Time Personalized Alignment with a Few User Preference Queries

Nov 04, 2025

This work addresses the high annotation cost and reliance on textual input in personalizing generative models via preference alignment. We propose an efficient alignment method that leverages only a few pairwise response comparisons at inference time. Our core innovation is modeling user feedback as consistent, noise-free ordinal preferences and integrating optimal arm identification theory from the logistic bandits framework to enable rapid personalized selection from a fixed candidate response pool. Crucially, the method eliminates the need for users to provide textual feedback—only 3–5 binary comparisons suffice to significantly improve output alignment with individual preferences. We validate its effectiveness across diverse tasks, including text summarization and image generation. Compared to baseline methods, our approach reduces query complexity by over 70% while maintaining or even improving alignment quality.

0 citationsRead paper

Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Models

Mar 27, 2025

This paper identifies probability distortion induced by local normalization during large language model decoding as the fundamental reason why top-k sampling underperforms nucleus (top-p) sampling. We introduce the novel concept of “local normalization distortion” and establish a unified formal framework for decoding strategies grounded in thermodynamics and ergodic theory, modeling top-k, nucleus, and temperature sampling as equilibrium states of ergodic systems. Through probabilistic measure perturbation analysis and information-theoretic metric modeling, we quantitatively characterize how this distortion degrades generation quality and diversity. Our work enables a paradigm shift from heuristic parameter tuning to principle-driven decoding algorithm design, yielding theoretically grounded, interpretable, and optimizable decoders—thereby providing a new foundation for both controllable text generation and AI-generated content detection. (149 words)

0 citationsRead paper

Policy Teaching via Data Poisoning in Learning from Human Preferences

Mar 13, 2025

This work investigates data poisoning attacks against human preference data in preference learning, aiming to steer model convergence toward a target policy π† via synthetically corrupted preference samples. We propose the first general theoretical framework for preference-learning-specific data poisoning, rigorously deriving tight upper and lower bounds on the number of poisoned samples required for policy teaching under both data-augmentation and fully synthetic attack settings. Through theoretical analysis and empirical evaluation, we uncover a fundamental disparity in poisoning robustness between RLHF and DPO: DPO’s direct optimization of the preference loss renders it significantly more vulnerable, requiring far fewer poisoned samples than RLHF to achieve successful policy teaching. Our results provide the first quantitative characterization of the vulnerability boundaries of mainstream preference learning paradigms, establishing both theoretical foundations and empirical evidence for designing robust alignment algorithms. (138 words)

0 citationsRead paper

Fairness-Aware Low-Rank Adaptation Under Demographic Privacy Constraints

Mar 07, 2025

This work addresses the inherent tension between fairness and privacy in LoRA-based fine-tuning. We propose a distributed fair fine-tuning framework that requires no access to sensitive attributes or their predictors. Methodologically, it integrates low-rank adaptation, sensitive information forgetting, adversarial training, and an orthogonality loss function within a collaborative training paradigm between model developers and fairness auditors, enforcing strict demographic privacy constraints. Our key contribution is breaking the conventional dependence on sensitive attributes in fair learning—achieving fairness-aware fine-tuning for the first time without any exposure of sensitive information. Experiments on CelebA and UTK-Face demonstrate that the orthogonality loss significantly reduces bias while preserving model utility; adversarial training effectively improves both false positive rate parity and demographic parity; and the overall framework achieves measurable fairness gains under strong privacy guarantees.

0 citationsRead paper

Recent publications

Latest Papers

Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMs

Nov 04, 2025

This work addresses the challenge of efficient learning and generalization for agents operating under stringent trajectory constraints—such as resource limitations or safety-critical requirements—during deployment. To this end, we propose a progressive curriculum learning framework. Methodologically, we introduce curriculum learning to trajectory-constrained reinforcement learning for the first time, employing a self-paced strategy that dynamically tightens constraints from loose to strict. We further integrate binary-tree MDP modeling, a multi-task navigation architecture, and LLM-based chain-of-thought (CoT) compression and inference acceleration at the token level. Experiments demonstrate substantial improvements in training efficiency and policy robustness, validate strong generalization across diverse constraint settings, and achieve significant CoT compression and inference speedup in LLM-driven reasoning.

0 citationsRead paper

Inference-Time Personalized Alignment with a Few User Preference Queries

Nov 04, 2025

This work addresses the high annotation cost and reliance on textual input in personalizing generative models via preference alignment. We propose an efficient alignment method that leverages only a few pairwise response comparisons at inference time. Our core innovation is modeling user feedback as consistent, noise-free ordinal preferences and integrating optimal arm identification theory from the logistic bandits framework to enable rapid personalized selection from a fixed candidate response pool. Crucially, the method eliminates the need for users to provide textual feedback—only 3–5 binary comparisons suffice to significantly improve output alignment with individual preferences. We validate its effectiveness across diverse tasks, including text summarization and image generation. Compared to baseline methods, our approach reduces query complexity by over 70% while maintaining or even improving alignment quality.

0 citationsRead paper

Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Models

Mar 27, 2025

This paper identifies probability distortion induced by local normalization during large language model decoding as the fundamental reason why top-k sampling underperforms nucleus (top-p) sampling. We introduce the novel concept of “local normalization distortion” and establish a unified formal framework for decoding strategies grounded in thermodynamics and ergodic theory, modeling top-k, nucleus, and temperature sampling as equilibrium states of ergodic systems. Through probabilistic measure perturbation analysis and information-theoretic metric modeling, we quantitatively characterize how this distortion degrades generation quality and diversity. Our work enables a paradigm shift from heuristic parameter tuning to principle-driven decoding algorithm design, yielding theoretically grounded, interpretable, and optimizable decoders—thereby providing a new foundation for both controllable text generation and AI-generated content detection. (149 words)

0 citationsRead paper

Policy Teaching via Data Poisoning in Learning from Human Preferences

Mar 13, 2025

This work investigates data poisoning attacks against human preference data in preference learning, aiming to steer model convergence toward a target policy π† via synthetically corrupted preference samples. We propose the first general theoretical framework for preference-learning-specific data poisoning, rigorously deriving tight upper and lower bounds on the number of poisoned samples required for policy teaching under both data-augmentation and fully synthetic attack settings. Through theoretical analysis and empirical evaluation, we uncover a fundamental disparity in poisoning robustness between RLHF and DPO: DPO’s direct optimization of the preference loss renders it significantly more vulnerable, requiring far fewer poisoned samples than RLHF to achieve successful policy teaching. Our results provide the first quantitative characterization of the vulnerability boundaries of mainstream preference learning paradigms, establishing both theoretical foundations and empirical evidence for designing robust alignment algorithms. (138 words)

0 citationsRead paper

Fairness-Aware Low-Rank Adaptation Under Demographic Privacy Constraints

Mar 07, 2025

This work addresses the inherent tension between fairness and privacy in LoRA-based fine-tuning. We propose a distributed fair fine-tuning framework that requires no access to sensitive attributes or their predictors. Methodologically, it integrates low-rank adaptation, sensitive information forgetting, adversarial training, and an orthogonality loss function within a collaborative training paradigm between model developers and fairness auditors, enforcing strict demographic privacy constraints. Our key contribution is breaking the conventional dependence on sensitive attributes in fair learning—achieving fairness-aware fine-tuning for the first time without any exposure of sensitive information. Experiments on CelebA and UTK-Face demonstrate that the orthogonality loss significantly reduces bias while preserving model utility; adversarial training effectively improves both false positive rate parity and demographic parity; and the overall framework achieves measurable fairness gains under strong privacy guarantees.

0 citationsRead paper