Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment
为了解决大语言模型难以适应多样化个人偏好的问题,本文提出了一种新的个性化群体相对策略优化框架P-GRPO,通过针对不同偏好组的历史奖励进行优势估计,从而更好地学习和对齐不同的用户偏好。
为了解决大语言模型难以适应多样化个人偏好的问题,本文提出了一种新的个性化群体相对策略优化框架P-GRPO,通过针对不同偏好组的历史奖励进行优势估计,从而更好地学习和对齐不同的用户偏好。
This work addresses the key challenge of balancing inference accuracy and error risk control under limited computational budgets in large language model reasoning. The authors reformulate the computational budget allocation problem as an adaptive stopping decision task under explicit risk constraints and propose a dynamic termination mechanism based on dual confidence thresholds. By parameterizing a lower threshold to early-exit hopeless samples and integrating a distribution-free risk control method to optimize the stopping policy, the approach ensures rigorous adherence to user-specified risk limits. Furthermore, an efficiency loss metric is introduced to select the optimal exit strategy across multiple criteria. Experimental results demonstrate that the proposed method significantly improves computational efficiency across diverse tasks and models while strictly satisfying prescribed risk upper bounds.
This work challenges the prevailing reliance on deterministic outputs in large language model (LLM) inference, which obscures inherent uncertainties, vulnerabilities, and safety risks while suppressing emergent capabilities and multi-path reasoning. To address this, the authors propose the “Stochastic CHAOS” framework, which treats output distribution variability as a core cognitive signal. By integrating multi-path sampling, non-deterministic attention mechanisms, and multi-sample evaluation, the framework systematically uncovers the true capabilities and latent risks of LLMs. Experimental results demonstrate that deterministic inference substantially underestimates both model performance and tail-end risks. In contrast, the proposed approach effectively recovers emergent phenomena, enhances reasoning accuracy, and reveals hidden safety hazards, thereby questioning conventional assumptions about reliability and reproducibility in LLM deployment.
This study addresses the challenge of modeling interpretable speech quality dimensions to characterize speaker styles in atypical and affective speech. We propose the first systematically defined set of seven phonatory quality primitives—e.g., clarity, glottalization, and breathiness—and train linear probes on frozen Wav2Vec 2.0 embeddings using the Speech Accessibility Project (SAP) dataset. Our method achieves strong interpretability and cross-domain generalizability: it attains state-of-the-art performance on SAP and, for the first time, enables zero-shot transfer across languages (English/Italian) and tasks (atypical vs. emotional speech), with average accuracy exceeding 82%. These results validate the framework’s efficacy as a universal, style-sensitive speech representation. The core contribution is the first interpretable, transferable, and multilingual-compatible speech quality dimension modeling framework.
To address the challenge of balancing privacy preservation and query efficiency in large-scale private database search, this paper proposes Wally—a scalable system achieving $(varepsilon,delta)$-differential privacy under high concurrency. Wally integrates four key techniques: (i) an adaptive dummy-query attenuation mechanism—where the number of dummy queries decreases with the number of honest users; (ii) Tor-based anonymous routing; (iii) randomized query submission timing; and (iv) lightweight homomorphic encryption. This design ensures that per-query privacy overhead asymptotically approaches zero as system scale increases. To our knowledge, Wally is the first system to enable efficient private search at the scale of ten million database entries and one million concurrent users. Experiments on a 3.2-million-entry dataset show that Wally achieves 7–28× higher QPS than Tiptoe while reducing communication overhead by 6.69–31×.
本文通过引入REVERSAL-BENCH,利用可调参数控制环境的可逆性及提供状态恢复验证机制,解决无外部重置强化学习中因不可逆事件导致的学习中断问题。
研究探讨了状态空间模型(SSMs)中门控机制对记忆与上下文学习的影响,通过理论和实验揭示了其在训练动态及泛化中的关键作用。
本文针对少样本跨域目标检测问题,提出使用软提示优化方法,在冻结预训练模型的情况下,通过少量连续提示符优化实现高效适应。
为解决视频描述评估难题,提出基于信息保真度的CapQuiz方法,通过多选题形式评估视频字幕质量。
为解决企业数据目录中列描述缺失和治理标签未分配的问题,Glyph系统通过多策略代理生成列描述并使用三种并行策略进行类型标注。