XL-DURel: Finetuning Sentence Transformers for Ordinal Word-in-Context Classification

2025-07-19

AI Summary

This work addresses ordinal contextual classification for multilingual word sense disambiguation (WSD), framing binary word-in-context (WiC) as a special case of ordinal WiC. Methodologically, it introduces angular distance in complex space as the ordinal ranking objective—first proposed for this task—and integrates it into a multilingual Sentence Transformer architecture, jointly optimizing regression and ordinal ranking losses. This unified framework eliminates task-form fragmentation by consistently modeling both ordinal and binary WiC under a single objective. Experiments demonstrate substantial improvements over state-of-the-art models on both multilingual ordinal WiC benchmarks and standard binary WiC tasks, confirming that generic ordinal optimization yields cross-task and cross-lingual generalization benefits. The approach establishes a novel paradigm for contextualized, multilingual lexical semantic modeling grounded in ordinal semantics.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

We propose XL-DURel, a finetuned, multilingual Sentence Transformer model optimized for ordinal Word-in-Context classification. We test several loss functions for regression and ranking tasks managing to outperform previous models on ordinal and binary data with a ranking objective based on angular distance in complex space. We further show that binary WiC can be treated as a special case of ordinal WiC and that optimizing models for the general ordinal task improves performance on the more specific binary task. This paves the way for a unified treatment of WiC modeling across different task formulations.

Problem

• Optimize multilingual Sentence Transformer for ordinal Word-in-Context classification
• Test loss functions for regression and ranking to outperform previous models
• Treat binary WiC as a special case of ordinal WiC for unified modeling

Innovation

• Multilingual Sentence Transformer finetuned for WiC
• Ranking objective based on angular distance
• Unified ordinal and binary WiC task modeling