Selected work

Representative Papers

Aurora: A Leverage-Aware Spectral Optimizer

Jun 26, 2026

This work addresses the issue in the Muon optimizer where highly anisotropic row norms in tall-and-skinny matrices—such as those in MLP projection layers—lead to severely imbalanced neuron updates, causing some neurons to receive negligible or ineffective gradients. To resolve this, the authors propose the Aurora optimizer, which introduces a row-normalization mechanism while preserving the geometric structure of the momentum matrix’s polar factor. Aurora is the first method to achieve uniformly scaled row-wise updates without compromising this intrinsic geometry—a limitation inherent in prior approaches that trade geometric fidelity for update uniformity. By integrating spectral optimization techniques with polar factor constraints, Aurora outperforms Muon in pretraining and achieves state-of-the-art performance among spectral optimizers on the modded-nanoGPT speedrun benchmark, with gains that become increasingly pronounced as the MLP expansion factor grows.

0 citationsRead paper

TildeOpen LLM: Leveraging Curriculum Learning to Achieve Equitable Language Representation

Mar 09, 2026

This work addresses the poor performance of large language models on most low-resource European languages, primarily caused by severe training data bias toward high-resource languages such as English. To mitigate this imbalance, the authors develop a 30-billion-parameter open-source large language model supporting 34 European languages and introduce a novel training paradigm that integrates curriculum learning with dynamic data upsampling. During pretraining, the model alternates between uniform and natural language data distributions to enhance representation learning for underrepresented language families—including Baltic, Uralic, and Slavic languages—without increasing model size or total training compute. The approach substantially improves cross-lingual performance parity, outperforming existing open-source models across multiple multilingual benchmarks. Human evaluations reveal up to a tenfold reduction in linguistic errors, and the code, model weights, and training pipeline have been publicly released on Hugging Face.

0 citationsRead paper

Localizing AI: Evaluating Open-Weight Language Models for Languages of Baltic States

Jan 07, 2025

This study addresses the challenge of securely deploying large language models (LLMs) for low-resource Baltic languages—Lithuanian, Latvian, and Estonian—in privacy- and security-sensitive domains such as government and defense. We systematically evaluate open-source, locally deployable multilingual LLMs—including Llama 3, Gemma 2, Phi, and NeMo—across machine translation, multiple-choice question answering, and free-text generation. We identify pervasive token-level hallucinations (average error rate ≥5%, i.e., one error per 20 tokens), demonstrating that high translation accuracy does not guarantee semantic reliability. Through FP16/INT4 precision analysis and a custom evaluation benchmark, we find Gemma 2 approaches commercial-model performance, yet all models exhibit critical deficiencies requiring language-specific optimization. Our work establishes the first empirical benchmark for LLM deployment in privacy-critical, low-resource language settings and provides concrete, actionable pathways for improving linguistic fidelity and trustworthiness.

0 citationsRead paper

Recent publications

Latest Papers

Aurora: A Leverage-Aware Spectral Optimizer

Jun 26, 2026

This work addresses the issue in the Muon optimizer where highly anisotropic row norms in tall-and-skinny matrices—such as those in MLP projection layers—lead to severely imbalanced neuron updates, causing some neurons to receive negligible or ineffective gradients. To resolve this, the authors propose the Aurora optimizer, which introduces a row-normalization mechanism while preserving the geometric structure of the momentum matrix’s polar factor. Aurora is the first method to achieve uniformly scaled row-wise updates without compromising this intrinsic geometry—a limitation inherent in prior approaches that trade geometric fidelity for update uniformity. By integrating spectral optimization techniques with polar factor constraints, Aurora outperforms Muon in pretraining and achieves state-of-the-art performance among spectral optimizers on the modded-nanoGPT speedrun benchmark, with gains that become increasingly pronounced as the MLP expansion factor grows.

0 citationsRead paper

TildeOpen LLM: Leveraging Curriculum Learning to Achieve Equitable Language Representation

Mar 09, 2026

This work addresses the poor performance of large language models on most low-resource European languages, primarily caused by severe training data bias toward high-resource languages such as English. To mitigate this imbalance, the authors develop a 30-billion-parameter open-source large language model supporting 34 European languages and introduce a novel training paradigm that integrates curriculum learning with dynamic data upsampling. During pretraining, the model alternates between uniform and natural language data distributions to enhance representation learning for underrepresented language families—including Baltic, Uralic, and Slavic languages—without increasing model size or total training compute. The approach substantially improves cross-lingual performance parity, outperforming existing open-source models across multiple multilingual benchmarks. Human evaluations reveal up to a tenfold reduction in linguistic errors, and the code, model weights, and training pipeline have been publicly released on Hugging Face.

0 citationsRead paper

Localizing AI: Evaluating Open-Weight Language Models for Languages of Baltic States

Jan 07, 2025

This study addresses the challenge of securely deploying large language models (LLMs) for low-resource Baltic languages—Lithuanian, Latvian, and Estonian—in privacy- and security-sensitive domains such as government and defense. We systematically evaluate open-source, locally deployable multilingual LLMs—including Llama 3, Gemma 2, Phi, and NeMo—across machine translation, multiple-choice question answering, and free-text generation. We identify pervasive token-level hallucinations (average error rate ≥5%, i.e., one error per 20 tokens), demonstrating that high translation accuracy does not guarantee semantic reliability. Through FP16/INT4 precision analysis and a custom evaluation benchmark, we find Gemma 2 approaches commercial-model performance, yet all models exhibit critical deficiencies requiring language-specific optimization. Our work establishes the first empirical benchmark for LLM deployment in privacy-critical, low-resource language settings and provides concrete, actionable pathways for improving linguistic fidelity and trustworthiness.

0 citationsRead paper