Institution profile

MATS

Research institutionnorthamerica · us
Research library15linked papers
Opportunities0open roles
Selected work

Representative Papers

Synthetic Persona Pretraining: Alignment from Token Zero

Aug 13, 2026

This work addresses the limitation of conventional language models, whose post-pretraining alignment mechanisms often result in superficial value adherence that degrades under out-of-distribution scenarios. To overcome this, the authors propose Synthetic Persona Pretraining (SPP), a novel approach that integrates value alignment at the very beginning of pretraining. SPP injects first-person reflective texts generated from a value constitution, enabling the assistant persona to internalize normative principles from token zero. This early-stage alignment is further reinforced through subsequent role-binding fine-tuning, achieving deep and robust value alignment. Experimental results demonstrate that SPP substantially enhances the model’s adherence to ethical guidelines, improves robustness against jailbreaking attempts, and maintains stable alignment in out-of-distribution moral dilemmas. Notably, introducing alignment during early pretraining yields superior performance without compromising general task capabilities.

0 citationsRead paper

Stealing Reasoning Traces from Proprietary LLM APIs

Aug 10, 2026

This work systematically uncovers a critical compatibility vulnerability in encrypted reasoning chains employed by large language model (LLM) providers to safeguard intellectual property. Despite encryption, reasoning blocks exhibit cross-session, cross-user, and cross-model interoperability flaws. The authors propose a novel decryption-based jailbreaking technique that leverages reverse engineering and cross-model reasoning block injection: by exploiting weaker models to decrypt the encrypted inference traces of stronger ones, the method reconstructs full reasoning processes without direct attacks. Empirical evaluation demonstrates successful extraction of reasoning chains from models by Anthropic, OpenAI, and Google, decrypting 315,320 publicly logged blocks, recovering 367 personally identifiable information (PII) instances and 182 credential sets, and enabling stealthy prompt injection that effectively bypasses existing anti-distillation and security mechanisms.

0 citationsRead paper

LangChoiceBench: Measuring and Explaining Programming-Language Choice in LLMs

Aug 06, 2026

This work addresses the pervasive over-reliance on Python by large language models (LLMs) in project-level code generation and the absence of systematic evaluation of their programming language selection behavior. We introduce LangChoiceBench, a benchmark spanning 28 projects across seven software domains, which enables the first systematic quantification of LLMs’ language preferences and diversity. Through manual analysis of 9,826 inference trajectories, we uncover novel failure modes, including “hallucinated evidence” and inconsistencies between recommended and implemented languages. Evaluation of 25 prominent LLMs reveals that most models select languages based on habitual bias rather than project requirements, with smaller models exhibiting even poorer language diversity, thereby highlighting a critical limitation of current LLMs in cross-language project generation.

0 citationsRead paper
Recent publications

Latest Papers

Synthetic Persona Pretraining: Alignment from Token Zero

Aug 13, 2026

This work addresses the limitation of conventional language models, whose post-pretraining alignment mechanisms often result in superficial value adherence that degrades under out-of-distribution scenarios. To overcome this, the authors propose Synthetic Persona Pretraining (SPP), a novel approach that integrates value alignment at the very beginning of pretraining. SPP injects first-person reflective texts generated from a value constitution, enabling the assistant persona to internalize normative principles from token zero. This early-stage alignment is further reinforced through subsequent role-binding fine-tuning, achieving deep and robust value alignment. Experimental results demonstrate that SPP substantially enhances the model’s adherence to ethical guidelines, improves robustness against jailbreaking attempts, and maintains stable alignment in out-of-distribution moral dilemmas. Notably, introducing alignment during early pretraining yields superior performance without compromising general task capabilities.

0 citationsRead paper

Stealing Reasoning Traces from Proprietary LLM APIs

Aug 10, 2026

This work systematically uncovers a critical compatibility vulnerability in encrypted reasoning chains employed by large language model (LLM) providers to safeguard intellectual property. Despite encryption, reasoning blocks exhibit cross-session, cross-user, and cross-model interoperability flaws. The authors propose a novel decryption-based jailbreaking technique that leverages reverse engineering and cross-model reasoning block injection: by exploiting weaker models to decrypt the encrypted inference traces of stronger ones, the method reconstructs full reasoning processes without direct attacks. Empirical evaluation demonstrates successful extraction of reasoning chains from models by Anthropic, OpenAI, and Google, decrypting 315,320 publicly logged blocks, recovering 367 personally identifiable information (PII) instances and 182 credential sets, and enabling stealthy prompt injection that effectively bypasses existing anti-distillation and security mechanisms.

0 citationsRead paper

LangChoiceBench: Measuring and Explaining Programming-Language Choice in LLMs

Aug 06, 2026

This work addresses the pervasive over-reliance on Python by large language models (LLMs) in project-level code generation and the absence of systematic evaluation of their programming language selection behavior. We introduce LangChoiceBench, a benchmark spanning 28 projects across seven software domains, which enables the first systematic quantification of LLMs’ language preferences and diversity. Through manual analysis of 9,826 inference trajectories, we uncover novel failure modes, including “hallucinated evidence” and inconsistencies between recommended and implemented languages. Evaluation of 25 prominent LLMs reveals that most models select languages based on habitual bias rather than project requirements, with smaller models exhibiting even poorer language diversity, thereby highlighting a critical limitation of current LLMs in cross-language project generation.

0 citationsRead paper