PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users

2026-05-13

AI Summary

This study investigates the performance discrepancy of personalized dialogue systems between real and simulated users, offering a systematic comparison between prompt-based personalization and preference-aligned fine-tuning via Preference-based Direct Preference Optimization (P-DPO). Through multi-turn blind evaluations involving 530 real users, longitudinal follow-ups two years later, and replication experiments with simulated users, the work reveals—for the first time—that simulated users significantly diverge from human behavior in individual consistency, topic selection, and feedback dynamics. The findings demonstrate that personalized fine-tuning substantially outperforms both generic models and prompt-based approaches, yet individualized fine-tuning yields only marginal gains over group-level tuning. Although simulated users can reproduce overall model rankings, they fall markedly short of real users in critical dimensions such as judgment consistency, highlighting their limitations for reliable evaluation.

Abstract

Personalisation is a standard feature of conversational AI systems used by millions; yet, the efficacy of personalisation methods is often evaluated in academic research using simulated users rather than real people. This raises questions about how users and their simulated counterparts differ in interaction patterns and judgements, as well as whether personalisation is best achieved through context-based prompting or weight-based fine-tuning. Here, in a large-scale within-subject experiment, we re-recruit 530 participants from 52 countries two years after they gave their preferences in the PRISM dataset (Kirk et al., 2024) to evaluate personalised and non-personalised language models in blinded multi-turn conversations. We find preference fine-tuning (P-DPO, Li et al., 2024) significantly outperforms both a generic model and personalised prompting but adapting to individual preference data yields marginal gains over training on pooled preferences from a diverse population. Beyond length biases, fine-tuning amplifies sycophancy and relationship-seeking behaviours that people reward in short-term evaluations but which may introduce deleterious long-term consequences. Replicating this within-subject experiment with simulated users recovers aggregate model hierarchies but simulators perform far below human self-consistency baselines for individual judgements, discuss different topics, exhibit amplified position biases, and produce feedback dynamics that diverge from humans.

Problem

• personalisation
• simulated users
• human evaluation
• fine-tuning
• conversational AI

Innovation

• personalised fine-tuning
• simulated users
• within-subject experiment
• preference alignment
• human-AI interaction