Selected work

Representative Papers

PRISM: A Geometric Risk Bound that Decomposes Drift into Scale, Shape, and Head

May 12, 2026

Existing methods struggle to diagnose representation drift in post-training variants of large language models—such as quantization or LoRA fine-tuning—as they can only assess performance degradation without identifying root causes or guiding mitigation. This work proposes PRISM, a method that leverages the linear output head and near-isometric backbone structure of LLMs to derive a closed-form upper bound on cross-entropy risk discrepancy. PRISM uniquely decomposes representation drift into three geometrically interpretable and independently measurable axes: scale, shape, and head. Notably, the shape component is differentiable and can be employed as a regularizer to mitigate catastrophic forgetting. Experiments show that PRISM achieves Spearman correlation coefficients of 0.820 and 0.831 in risk ranking for quantized and LoRA-adapted models, respectively, across two model families and five benchmarks; moreover, shape-based regularization outperforms experience replay in alleviating downstream forgetting.

0 citationsRead paper

On Calibration of Large Language Models: From Response To Capability

Feb 14, 2026

This work addresses a critical limitation in existing calibration methods for large language models (LLMs), which focus on the correctness of individual responses and often fail to reflect the model’s overall task-solving capability, leading to a misalignment between confidence and actual performance. To bridge this gap, we propose a novel paradigm—capability calibration—that estimates the expected accuracy of an LLM on a given query, thereby shifting the focus from response-level to task-level reliability assessment. We formally distinguish capability calibration from traditional response calibration, develop a theoretical framework grounded in the stochasticity of LLM decoding, and systematically evaluate various confidence estimation methods under this new paradigm. Experiments demonstrate that capability calibration substantially improves the accuracy of pass@$k$ prediction and enhances the efficiency of reasoning resource allocation, offering a more reliable foundation for downstream applications.

0 citationsRead paper

Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?

May 23, 2025

This study identifies an implicit language bias in large reasoning models (LRMs): when processing multilingual inputs, LRMs default to high-resource languages—particularly English—for internal reasoning, severely degrading performance on low-resource language tasks. To systematically investigate this, we design a multidimensional, controllable evaluation framework covering MMMLU, MATH-500, CulturalBench, and LMSYS-toxic, integrating reasoning-path tracing and cross-lingual attribution analysis. We empirically demonstrate that enforcing same-language reasoning—though it reduces general reasoning capability (especially for low-resource languages)—significantly improves cultural alignment and language-specific accuracy in safety evaluation. Crucially, this work is the first to empirically establish “reasoning-language–input-language mismatch” as a fundamental bottleneck to multilingual fairness. Our findings provide both theoretical grounding and methodological tools for developing language-neutral LRMs.

0 citationsRead paper

Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors

Apr 27, 2025

This paper addresses the fundamental trade-off between helpfulness and harmlessness in large language models (LLMs). We propose a novel preference vector framework that decouples multi-objective alignment into independent preference modeling and runtime vector fusion. Specifically, we first train separate single-objective policies—for helpfulness, safety, etc.—and then extract behavior-offset preference vectors from them; these vectors are dynamically combined via task-aware arithmetic weighting during inference. Compared to existing methods, our framework significantly mitigates objective conflicts, enabling fine-grained, scalable, and user-controllable preference adjustment. Crucially, it supports zero-shot integration of new preferences without additional fine-tuning, simultaneously enhancing helpfulness while strictly preserving safety. Moreover, the framework provides smooth, interpretable, and controllable trade-offs across competing objectives—achieving both robust alignment and operational flexibility.

0 citationsRead paper

None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering

Mar 03, 2025

Large language models (LLMs) exhibit poor calibration and systematic failure in multiple-choice questions containing a “None of the Above” (NA) option, revealing fundamental deficits in meta-cognitive rejection and uncertainty awareness. Method: We conduct controlled experiments across 28 LLMs on the MMLU benchmark, integrating confidence-score analysis with cross-scale and cross-disciplinary attribution to isolate NA-specific performance degradation. Contribution/Results: We identify, for the first time, a catastrophic 30–50% average accuracy drop when NA is the correct answer—demonstrating a severe, domain-dependent deficit in option negation capability. Performance decline varies markedly by discipline: only −14.6% in mathematical reasoning versus −48.1% in business ethics, where uncertainty recognition is critical. These findings expose a core limitation in LLMs’ ability to withhold responses under epistemic uncertainty. Our work establishes a novel evaluation paradigm for uncertainty-aware reasoning and provides key empirical evidence for assessing meta-cognitive refusal capacity in foundation models.

0 citationsRead paper

Recent publications

Latest Papers

PRISM: A Geometric Risk Bound that Decomposes Drift into Scale, Shape, and Head

May 12, 2026

Existing methods struggle to diagnose representation drift in post-training variants of large language models—such as quantization or LoRA fine-tuning—as they can only assess performance degradation without identifying root causes or guiding mitigation. This work proposes PRISM, a method that leverages the linear output head and near-isometric backbone structure of LLMs to derive a closed-form upper bound on cross-entropy risk discrepancy. PRISM uniquely decomposes representation drift into three geometrically interpretable and independently measurable axes: scale, shape, and head. Notably, the shape component is differentiable and can be employed as a regularizer to mitigate catastrophic forgetting. Experiments show that PRISM achieves Spearman correlation coefficients of 0.820 and 0.831 in risk ranking for quantized and LoRA-adapted models, respectively, across two model families and five benchmarks; moreover, shape-based regularization outperforms experience replay in alleviating downstream forgetting.

0 citationsRead paper

On Calibration of Large Language Models: From Response To Capability

Feb 14, 2026

This work addresses a critical limitation in existing calibration methods for large language models (LLMs), which focus on the correctness of individual responses and often fail to reflect the model’s overall task-solving capability, leading to a misalignment between confidence and actual performance. To bridge this gap, we propose a novel paradigm—capability calibration—that estimates the expected accuracy of an LLM on a given query, thereby shifting the focus from response-level to task-level reliability assessment. We formally distinguish capability calibration from traditional response calibration, develop a theoretical framework grounded in the stochasticity of LLM decoding, and systematically evaluate various confidence estimation methods under this new paradigm. Experiments demonstrate that capability calibration substantially improves the accuracy of pass@$k$ prediction and enhances the efficiency of reasoning resource allocation, offering a more reliable foundation for downstream applications.

0 citationsRead paper

Language Matters: How Do Multilingual Input and Reasoning Paths Affect Large Reasoning Models?

May 23, 2025

This study identifies an implicit language bias in large reasoning models (LRMs): when processing multilingual inputs, LRMs default to high-resource languages—particularly English—for internal reasoning, severely degrading performance on low-resource language tasks. To systematically investigate this, we design a multidimensional, controllable evaluation framework covering MMMLU, MATH-500, CulturalBench, and LMSYS-toxic, integrating reasoning-path tracing and cross-lingual attribution analysis. We empirically demonstrate that enforcing same-language reasoning—though it reduces general reasoning capability (especially for low-resource languages)—significantly improves cultural alignment and language-specific accuracy in safety evaluation. Crucially, this work is the first to empirically establish “reasoning-language–input-language mismatch” as a fundamental bottleneck to multilingual fairness. Our findings provide both theoretical grounding and methodological tools for developing language-neutral LRMs.

0 citationsRead paper

Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors

Apr 27, 2025

This paper addresses the fundamental trade-off between helpfulness and harmlessness in large language models (LLMs). We propose a novel preference vector framework that decouples multi-objective alignment into independent preference modeling and runtime vector fusion. Specifically, we first train separate single-objective policies—for helpfulness, safety, etc.—and then extract behavior-offset preference vectors from them; these vectors are dynamically combined via task-aware arithmetic weighting during inference. Compared to existing methods, our framework significantly mitigates objective conflicts, enabling fine-grained, scalable, and user-controllable preference adjustment. Crucially, it supports zero-shot integration of new preferences without additional fine-tuning, simultaneously enhancing helpfulness while strictly preserving safety. Moreover, the framework provides smooth, interpretable, and controllable trade-offs across competing objectives—achieving both robust alignment and operational flexibility.

0 citationsRead paper

None of the Above, Less of the Right: Parallel Patterns between Humans and LLMs on Multi-Choice Questions Answering

Mar 03, 2025

Large language models (LLMs) exhibit poor calibration and systematic failure in multiple-choice questions containing a “None of the Above” (NA) option, revealing fundamental deficits in meta-cognitive rejection and uncertainty awareness. Method: We conduct controlled experiments across 28 LLMs on the MMLU benchmark, integrating confidence-score analysis with cross-scale and cross-disciplinary attribution to isolate NA-specific performance degradation. Contribution/Results: We identify, for the first time, a catastrophic 30–50% average accuracy drop when NA is the correct answer—demonstrating a severe, domain-dependent deficit in option negation capability. Performance decline varies markedly by discipline: only −14.6% in mathematical reasoning versus −48.1% in business ethics, where uncertainty recognition is critical. These findings expose a core limitation in LLMs’ ability to withhold responses under epistemic uncertainty. Our work establishes a novel evaluation paradigm for uncertainty-aware reasoning and provides key empirical evidence for assessing meta-cognitive refusal capacity in foundation models.

0 citationsRead paper