OdysSim: Building Foundation Models for Human Behavior Simulation

2026-06-12

AI Summary

This work addresses the significant Sim2Real gap in current large language models (LLMs), whose behaviors have become overly homogeneous due to an excessive focus on usefulness, limiting their ability to authentically simulate human behavior. The authors propose the first systematic research paradigm for behavioral foundation models, introducing the SOUL framework—a five-dimensional capability taxonomy—and unifying 62 datasets across 23 tasks. They develop OSim, an 8B-parameter open model trained end-to-end via large-scale corpus reconstruction, intermediate training, task-oriented reinforcement learning, expert distillation, and reward-hacking detection. Key contributions include the SOUL capability taxonomy, the SOUL-Index benchmark, and novel insights into reward hacking in LLM-as-a-judge settings, along with mitigation strategies. OSim achieves top performance on 8 of 23 tasks, demonstrates marked improvements in dialogue and social interaction, and attains a 93.2% zero-shot response alignment rate on τ-bench, closely approaching human-level performance (93.5%).

Abstract

Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation. Yet helpfulness-driven post-training pulls them toward a homogeneous, overly agreeable assistant register, creating a behavioral Sim2Real gap. We present OdysSim, the largest open systematic investigation of behavioral foundation models, i.e., models trained to simulate human behavior at scale. We propose SOUL, a taxonomy of five capability axes (CONV, SS, COG, ROLE, EVAL) that unifies 62 datasets and 23 benchmark tasks under one framework. Specifically, we curate the OdysSim corpus (21.4M interactions, 10B tokens, retrofitted with back-generated social contexts), construct the SOUL-Index benchmark, and develop an end-to-end training recipe combining midtraining, task-specific RL, and expert distillation. The resulting open 8B OSim model ranks first or tied-first on 8 of 23 tasks, outperforming any individual frontier model by this count, with the strongest gains on conversational and social tasks. Its outputs are also more human-like in length, formatting, and word choice, and it transfers zero-shot to out-of-distribution user simulation on $τ$-bench, nearly matching real users on reaction alignment (93.2 vs. 93.5). We further show that LLM-as-judge RL induces reward-hacking patterns, and that our detectors can mitigate them during post-training. Together, our findings suggest that behavioral foundation models require rethinking the LLM training paradigm. We release all artifacts to support future research.

Problem

• human behavior simulation
• Sim2Real gap
• behavioral foundation models
• large language models
• social simulation

Innovation

• behavioral foundation models
• human behavior simulation
• SOUL taxonomy
• reward hacking mitigation
• zero-shot transfer