Multilinguality as Sense Adaptation

2026-01-15

AI Summary

This work proposes SENSIA, a novel approach to multilingual adaptation that explicitly incorporates sense-level alignment to address the challenge of efficiently leveraging limited target-language resources. Unlike conventional multilingual models that rely on shared parameters and large-scale data for semantic alignment, SENSIA aligns sense-mixture representations with contextual representations on parallel corpora and jointly optimizes target-language modeling loss to preserve generation fluency. Built upon the Backpack language model, the method integrates sense-mixture alignment, symmetric cross-lingual representation alignment, and language modeling in a unified training framework. Experiments across four typologically diverse languages demonstrate that SENSIA achieves performance comparable to monolingual models trained from scratch while using only one-half to one-fourth of the target-language data, substantially outperforming existing alignment strategies.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

We approach multilinguality as sense adaptation: aligning latent meaning representations across languages rather than relying solely on shared parameters and scale. In this paper, we introduce SENse-based Symmetric Interlingual Alignment (SENSIA), which adapts a Backpack language model from one language to another by explicitly aligning sense-level mixtures and contextual representations on parallel data, while jointly training a target-language language modeling loss to preserve fluency. Across benchmarks on four typologically diverse languages, SENSIA generally outperforms comparable multilingual alignment methods and achieves competitive accuracy against monolingual from-scratch baselines while using 2-4x less target-language data. Analyses of learned sense geometry indicate that local sense topology and global structure relative to English are largely preserved, and ablations show that the method is robust in terms of design and scale.

Problem

• multilinguality
• sense adaptation
• semantic alignment
• cross-lingual representation
• language modeling

Innovation

• sense adaptation
• interlingual alignment
• multilingual language modeling
• latent meaning representation
• parallel data