Soft Token Alignment for Cross-Lingual Reasoning

2026-06-24

AI Summary

This work addresses the inconsistency in reasoning exhibited by multilingual large language models when presented with semantically equivalent prompts in different languages, a problem often exacerbated by discrete tokenization. To mitigate this issue, the authors propose SOLAR, a novel approach that introduces soft tokens—probabilistic weighted combinations of lexical embeddings—as continuous semantic representations during supervised fine-tuning. By aligning multilingual soft token spaces through English as a pivot, SOLAR reduces reliance on specific vocabularies or writing systems while preserving cross-lingually shared reasoning structures. Experimental results demonstrate that SOLAR significantly outperforms baseline methods across four multilingual reasoning benchmarks, achieving gains of up to 17.7 percentage points, with particularly pronounced improvements for low-resource languages. The method also substantially enhances cross-lingual representational similarity.

Abstract

Multilingual large language models often produce inconsistent reasoning and answers for semantically equivalent prompts in different languages. Prior work suggests that intermediate representations can be relatively language-agnostic, but generation becomes increasingly language-specific as models commit to discrete output tokens. This is problematic because language-specific lexical choices can cause semantically equivalent reasoning paths to diverge across languages. These divergences motivate searching for a cross-lingual alignment signal that is less tied to any single vocabulary item or script. We propose SOLAR, an auxiliary objective for supervised fine-tuning that aligns soft-token representations across languages, using English as a pivot. Soft tokens are probability-weighted mixtures over the vocabulary embeddings, yielding continuous representations that can aggregate information from semantically related tokens across languages. We then align each non-English soft-token summary to its English counterpart in the shared embedding space. Across four multilingual reasoning benchmarks, SOLAR improves accuracy by up to +17.7 points over the base model and +3.8 over standard supervised fine-tuning, with the largest gains on low-resource languages. SOLAR also strengthens final-layer cross-lingual similarity and substantially reduces language-cluster separability, suggesting that aligning soft-token representations helps preserve shared semantic structure during multilingual reasoning.

Problem

• cross-lingual reasoning
• multilingual large language models
• language-specific generation
• semantic inconsistency
• token alignment

Innovation

• soft token alignment
• cross-lingual reasoning
• multilingual LLMs
• supervised fine-tuning
• language-agnostic representation