Selected work

Representative Papers

MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages

Jul 01, 2026

This work addresses the scarcity of non-English open-source pretraining corpora, which hinders the development of multilingual large language models. To overcome this limitation, the authors propose a machine translation–based synthesis approach that leverages the high-quality Nemotron-CC corpus and employs both Tower+ and OPUS-MT/HPLT-MT systems to generate a sentence-aligned parallel corpus spanning 36 European languages and approximately 4.8 trillion tokens—the first large-scale, open, multi-system-fused multilingual pretraining dataset of its kind. Experimental results demonstrate that, under a fixed budget of 100 billion tokens, models trained on this synthetic data achieve a ~15% performance gain over the native-data baseline HPLT 2.0; moreover, they reach equivalent final performance using only 72% of the token budget, substantially reducing reliance on scarce native multilingual data and exposing limitations in current evaluation benchmarks.

0 citationsRead paper

Agents Learn Their Runtime: Interpreter Persistence as Training-Time Semantics

Mar 01, 2026

This work addresses a critical yet overlooked issue in tool-augmented large language models: inconsistent semantics of interpreter state persistence between training and deployment, which can lead to execution errors or severe inefficiencies. For the first time, state persistence is explicitly modeled as a first-order semantic property of training data. The authors construct paired trajectories on the Opaque Knapsack task that differ only in state persistence behavior and perform a 2×2 cross-fine-tuning and evaluation study on Qwen3-8B. Results demonstrate that aligning state persistence between training and deployment substantially improves performance—mismatches incur up to 80% error rates or 3.5× redundant token consumption, while solution quality remains largely unaffected. This study reveals the pivotal role of state persistence in shaping agent reasoning paths, stability, and token efficiency.

0 citationsRead paper

MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources

Sep 29, 2025

To address legal risks and performance trade-offs arising from indiscriminate web crawling in LLM pretraining, this paper proposes a license-first data curation paradigm to construct an open, traceable, and legally compliant large-scale pretraining corpus. Methodologically, we design a risk-mitigating data provenance framework integrating public-domain, permissively licensed, and low-risk sources; implement a multi-stage pipeline featuring license-aware filtering, dual safety/quality screening, and domain-aware mixing; and incorporate instruction tuning, reasoning-oriented data augmentation, and controllable synthetic data generation. Evaluated on models ranging from 130M to 1.7B parameters trained on 50B–300B tokens, our corpus achieves performance surpassing FineWeb-Edu and approaching DCLM, with notable gains in mathematical reasoning and code generation. This work establishes a new benchmark for lawful, transparent, and reproducible LLM training.

0 citationsRead paper

EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detection

Jun 11, 2025

Existing speech emotion recognition (SER) datasets suffer from coarse emotional granularity, high privacy risks, and substantial performative bias, lacking psychologically validated, trustworthy benchmarks. To address these limitations, this work introduces the first fine-grained, privacy-preserving SER benchmark: built upon 4,500+ hours of synthetic speech (11 voices × 40 emotions × 4 languages), it features dual-dimensional expert-validated annotations—comprising 40 discrete emotion categories and intensity levels—covering low-frequency, sensitive emotions (e.g., shame, awe). Methodologically, we propose scenario-driven script generation, a multi-tier collaborative annotation protocol, and Empathic Insight Voice (EIV), a novel neural architecture. Experiments demonstrate that EIV achieves unprecedented agreement with human experts. Furthermore, ablation studies reveal significantly higher accuracy for high-arousal emotions (e.g., anger) versus low-arousal states (e.g., concentration), providing critical empirical evidence for SER interpretability and psychological validity.

0 citationsRead paper

EmoNet-Face: An Expert-Annotated Benchmark for Synthetic Emotion Recognition

May 26, 2025

Existing visual/visual-language affect recognition benchmarks suffer from three critical limitations: (1) a narrow affective spectrum—incapable of capturing nuanced states such as bitterness or euphoria; (2) ambiguous inter-class boundaries—e.g., shame vs. embarrassment; and (3) severe data biases—including pervasive facial occlusion and insufficient demographic diversity. To address these, we introduce EmoNet Face—the first high-fidelity, fine-grained affect recognition benchmark tailored for synthetic images. Our approach comprises: (i) a cognitively grounded 40-category affect taxonomy ensuring perceptual discriminability; (ii) controllable face modeling to generate large-scale, fully visible, demographically balanced AI-rendered faces; and (iii) a multi-expert collaborative annotation protocol coupled with fairness-aware data balancing. We release three complementary sub-datasets and the EmpathicInsight-Face model, which achieves human-expert-level annotation consistency on EmoNet Face (Cohen’s κ = 0.89).

0 citationsRead paper

Recent publications

Latest Papers

MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages

Jul 01, 2026

This work addresses the scarcity of non-English open-source pretraining corpora, which hinders the development of multilingual large language models. To overcome this limitation, the authors propose a machine translation–based synthesis approach that leverages the high-quality Nemotron-CC corpus and employs both Tower+ and OPUS-MT/HPLT-MT systems to generate a sentence-aligned parallel corpus spanning 36 European languages and approximately 4.8 trillion tokens—the first large-scale, open, multi-system-fused multilingual pretraining dataset of its kind. Experimental results demonstrate that, under a fixed budget of 100 billion tokens, models trained on this synthetic data achieve a ~15% performance gain over the native-data baseline HPLT 2.0; moreover, they reach equivalent final performance using only 72% of the token budget, substantially reducing reliance on scarce native multilingual data and exposing limitations in current evaluation benchmarks.

0 citationsRead paper

Agents Learn Their Runtime: Interpreter Persistence as Training-Time Semantics

Mar 01, 2026

This work addresses a critical yet overlooked issue in tool-augmented large language models: inconsistent semantics of interpreter state persistence between training and deployment, which can lead to execution errors or severe inefficiencies. For the first time, state persistence is explicitly modeled as a first-order semantic property of training data. The authors construct paired trajectories on the Opaque Knapsack task that differ only in state persistence behavior and perform a 2×2 cross-fine-tuning and evaluation study on Qwen3-8B. Results demonstrate that aligning state persistence between training and deployment substantially improves performance—mismatches incur up to 80% error rates or 3.5× redundant token consumption, while solution quality remains largely unaffected. This study reveals the pivotal role of state persistence in shaping agent reasoning paths, stability, and token efficiency.

0 citationsRead paper

MixtureVitae: Open Web-Scale Pretraining Dataset With High Quality Instruction and Reasoning Data Built from Permissive-First Text Sources

Sep 29, 2025

To address legal risks and performance trade-offs arising from indiscriminate web crawling in LLM pretraining, this paper proposes a license-first data curation paradigm to construct an open, traceable, and legally compliant large-scale pretraining corpus. Methodologically, we design a risk-mitigating data provenance framework integrating public-domain, permissively licensed, and low-risk sources; implement a multi-stage pipeline featuring license-aware filtering, dual safety/quality screening, and domain-aware mixing; and incorporate instruction tuning, reasoning-oriented data augmentation, and controllable synthetic data generation. Evaluated on models ranging from 130M to 1.7B parameters trained on 50B–300B tokens, our corpus achieves performance surpassing FineWeb-Edu and approaching DCLM, with notable gains in mathematical reasoning and code generation. This work establishes a new benchmark for lawful, transparent, and reproducible LLM training.

0 citationsRead paper

EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detection

Jun 11, 2025

Existing speech emotion recognition (SER) datasets suffer from coarse emotional granularity, high privacy risks, and substantial performative bias, lacking psychologically validated, trustworthy benchmarks. To address these limitations, this work introduces the first fine-grained, privacy-preserving SER benchmark: built upon 4,500+ hours of synthetic speech (11 voices × 40 emotions × 4 languages), it features dual-dimensional expert-validated annotations—comprising 40 discrete emotion categories and intensity levels—covering low-frequency, sensitive emotions (e.g., shame, awe). Methodologically, we propose scenario-driven script generation, a multi-tier collaborative annotation protocol, and Empathic Insight Voice (EIV), a novel neural architecture. Experiments demonstrate that EIV achieves unprecedented agreement with human experts. Furthermore, ablation studies reveal significantly higher accuracy for high-arousal emotions (e.g., anger) versus low-arousal states (e.g., concentration), providing critical empirical evidence for SER interpretability and psychological validity.

0 citationsRead paper

EmoNet-Face: An Expert-Annotated Benchmark for Synthetic Emotion Recognition

May 26, 2025

Existing visual/visual-language affect recognition benchmarks suffer from three critical limitations: (1) a narrow affective spectrum—incapable of capturing nuanced states such as bitterness or euphoria; (2) ambiguous inter-class boundaries—e.g., shame vs. embarrassment; and (3) severe data biases—including pervasive facial occlusion and insufficient demographic diversity. To address these, we introduce EmoNet Face—the first high-fidelity, fine-grained affect recognition benchmark tailored for synthetic images. Our approach comprises: (i) a cognitively grounded 40-category affect taxonomy ensuring perceptual discriminability; (ii) controllable face modeling to generate large-scale, fully visible, demographically balanced AI-rendered faces; and (iii) a multi-expert collaborative annotation protocol coupled with fairness-aware data balancing. We release three complementary sub-datasets and the EmpathicInsight-Face model, which achieves human-expert-level annotation consistency on EmoNet Face (Cohen’s κ = 0.89).

0 citationsRead paper