Personalizing MLLMs via Reinforced Multimodal Reference Game

2026-06-27

AI Summary

This work addresses the susceptibility of multimodal large language models to distracting contextual cues when generating personalized concept descriptions, which often leads to inaccurate or non-unique identification of target concepts. To mitigate this issue, the paper introduces the Reinforced Reference Game (RRG) framework—the first approach to integrate reinforcement learning with multimodal reference games. Within this framework, the model simultaneously assumes the roles of speaker and listener in a contrastive game setting. A verifiable contrastive reward mechanism is established using hard positive and hard negative examples, guiding the model to produce precise, interference-free, and highly discriminative descriptions. Evaluated on three personalized benchmarks, the proposed method significantly outperforms existing approaches, demonstrating superior generalization capability and descriptive discriminability.

Abstract

Personalizing Multimodal Large Language Models (MLLMs) aims to recognize users' unique concepts from visual data and provide personalized responses. Although prior work has shown the benefit of concept descriptions and reasoning for this task, MLLM descriptions often include information, such as state and context, that does not help and may in fact hinder the unique identification of the target concept among other visually similar items. Effective descriptions of personal concepts should instead be accurate, discriminative, and free of distracting details. To achieve such descriptions, we introduce Reinforced Reference Game (RRG), a learning framework that promotes discriminative descriptions through a novel reinforced multimodal reference game. The MLLM plays both the roles of speaker and listener in a contrastive game setting, whose goal is to effectively communicate discriminative information about a target concept. Our approach formulates a verifiable contrastive reward over hard positives (dissimilar views of the same concept) and hard negatives (visually similar but different concepts). Empirically, RRG achieves state-of-the-art across multiple tasks on three personalization benchmarks. RRG generalizes to unseen domains and outperforms existing methods based on concept descriptions and personalization-specific RL frameworks. We will release code and models in the project page.

Problem

• Personalization
• Multimodal Large Language Models
• Discriminative Description
• Visual Concept Identification
• Reference Game

Innovation

• Reinforced Reference Game
• Multimodal Large Language Models
• Personalization
• Contrastive Reward
• Discriminative Description