SocialPersona: Benchmarking Personalized Profiling and Response with Multimodal Social-Media Context

2026-06-25

AI Summary

This work addresses the limitations of existing personalized language models, which predominantly rely on explicit user preferences and struggle to infer authentic interests from naturally occurring multimodal social media traces. To bridge this gap, we introduce SocialPersona—the first benchmark for personalized evaluation grounded in real longitudinal social media data encompassing text, images, and timestamps. SocialPersona enables structured user profiling and personalized dialogue generation by distinguishing between stable and recent interests, while revealing the complementary roles of textual and visual signals in preference inference. Experiments demonstrate that current multimodal large language models can recognize broad interests but exhibit limited capability in modeling fine-grained and recent preferences; performance further degrades when leveraging these profiles for dialogue generation, underscoring long-term cross-modal user modeling as a critical open challenge.

Abstract

Personalized language-model assistants are often evaluated through a memory lens: can a model recall preferences users have explicitly stated in dialogue? More comprehensive personalization demands a harder capability -- inferring what users care about from the multimodal traces they naturally leave behind. We introduce SocialPersona, a benchmark for evaluating whether multimodal large language models (MLLMs) can recover revealed preferences from longitudinal social-media timelines and use them in dialogue. Built from longitudinal timelines of 171 everyday, non-promotional social-media users, SocialPersona contains text, images, timestamps, and 2,597 human-verified preference tags across seven interest domains, separating stable interests from recent interests. It supports two tasks: constructing structured user profiles from multimodal context and generating responses aligned with inferred profiles. Experiments with proprietary and open-weight MLLMs show that models can identify broad interest domains, yet their performance drops on fine-grained and recent interests and degrades further when inferred profiles must be used to personalize dialogue. Together with evidence that text and images provide complementary preference signals, these results indicate that robust cross-modal, long-horizon user modeling remains a key challenge, and that SocialPersona can help measure and advance progress toward assistants that infer and act on revealed preferences.

Problem

• personalized profiling
• multimodal social-media context
• revealed preferences
• user modeling
• long-horizon inference

Innovation

• multimodal large language models
• personalized profiling
• revealed preferences
• social-media timeline
• user modeling