OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing

2026-06-14

AI Summary

This work addresses the challenge of perceptual shortcut learning and poor interpretability in multimodal large language models (MLLMs) for tasks such as emotion recognition, which often stems from sparse rewards and a scarcity of high-quality chain-of-thought annotations. The authors propose OmniOPSD, a novel framework that leverages evidence-aware reasoning generated by a teacher model not as an imitation target but as privileged context during training. By integrating policy self-distillation with token-level dense supervision, the student model learns from its own reasoning trajectories without requiring access to explicit reasoning chains, human labels, or closed-source models. This approach substantially enhances model transparency and generalization, achieving state-of-the-art performance with an average score of 84.19 on MER-UniBench. Ablation studies further confirm the efficacy of the rationale-guided mechanism.

Abstract

Reinforcement learning for multimodal large language models (MLLMs) is often hindered by severe reward sparsity in complex reasoning tasks. This challenge is particularly pronounced in human-centered scenarios involving states, emotions, intentions, and behaviors, where heterogeneous multimodal signals and subjective human factors make high-quality chain-of-thought (CoT) annotations expensive and difficult to obtain. Although many multimodal datasets provide expert-annotated ground-truth labels, directly using these labels for supervised fine-tuning may encourage shortcut learning in multimodal perception and provides limited transparency for safety-critical human--AI interaction. To address these limitations, we propose OmniOPSD, a Rationale-Privileged On-Policy Self-Distillation framework that uses frontier-generated rationales as teacher-side privileged evidence rather than student imitation targets. OmniOPSD uses frontier-generated evidence-aware rationales only as training-time privileged evidence context for a local teacher. The student samples its own rollout from the original multimodal input, while the rationale-privileged teacher scores the same tokens and provides dense token-level supervision. Thus, the student learns on its own trajectory distribution without directly imitating frontier-model completions, and inference requires no labels, rationales, CoT annotations, or closed-source model access. Experiments on MER-UniBench show that OmniOPSD achieves state-of-the-art performance with an average score of $84.19$, and ablations further support the value of rationale-privileged teacher guidance.

Problem

• reward sparsity
• affective computing
• multimodal large language models
• chain-of-thought annotation
• shortcut learning

Innovation

• On-Policy Self-Distillation
• Rationale-Privileged Learning
• Multimodal Large Language Models
• Affective Computing
• Token-Level Supervision