LOPA: Enhancing Spoken Language Assessment via Latent Ordinal Prototype Alignment

2026-06-30

AI Summary

This work addresses the limitations of existing multimodal large language models in spoken language assessment, which often neglect the sequential structure inherent in language acquisition and rely heavily on extensive model fine-tuning. To overcome these issues, the authors propose a novel fine-tuning-free paradigm that explicitly embeds ordinal geometric priors into the latent space through Latent Ordinal Prototype Alignment (LOPA) and adaptively extracts multi-level representations from a frozen Whisper encoder via a Semantic Anchored Layer Routing (SALR) mechanism. This approach yields interpretable evaluations aligned with human scoring rubrics, achieving a root mean square error (RMSE) of 0.361 on spoken proficiency scoring—comparable to billion-parameter systems—while substantially reducing computational overhead.

Abstract

Fueled by increasing model scale and multimodal inputs, Multimodal Large Language Models (MLLMs) have emerged as a promising paradigm for Spoken Language Assessment (SLA). While effective, this paradigm often overlooks the intrinsic ordinal structure of language acquisition. This paper works around the necessity of large-scale MLLMs by introducing Latent Ordinal Prototype Alignment (LOPA) for SLA, a prototype-based regularizer that enforces an ordinal geometric prior directly on the latent space. Coupled with Semantic-Anchored Layer Routing (SALR), which adaptively harvests multi-depth representations from a frozen Whisper encoder, our framework achieves an RMSE of 0.361. This performance rivals billion-parameter systems without the need for LLM-based fine-tuning. Further analysis reveals that SALR's synergy with LOPA offers interpretable, criterion-aligned preferences, thereby supporting an efficient and ordinal-aware modeling alternative to current scaling-centric models for SLA.

Problem

• Spoken Language Assessment
• ordinal structure
• Multimodal Large Language Models
• latent space
• language acquisition

Innovation

• Latent Ordinal Prototype Alignment
• Spoken Language Assessment
• Ordinal Structure
• Semantic-Anchored Layer Routing
• Multimodal Large Language Models