Incoherent Values? Probing LLM Preferences Through Parametric Variation

2026-06-19

AI Summary

This study investigates whether large language models exhibit stable and consistent value preferences, a key requirement for their deployment as trustworthy autonomous agents. To this end, the authors propose a novel parameterized evaluation framework that systematically quantifies preference inconsistency across diverse scenarios through forced-choice tasks and preference inference analyses, comparing settings with and without explicit reasoning at inference time. Experimental results reveal that even state-of-the-art models display significant value conflicts, and preference coherence does not automatically emerge with increased model capability. However, incorporating a reasoning process substantially reduces such inconsistencies. The proposed framework establishes a quantifiable benchmark and provides a foundation for reward signals aimed at developing more coherent and reliable AI agents.

Abstract

To trust another autonomous entity -- human or AI -- it helps to know that how it acts given one set of reasons is at least somewhat predictive of how it would act under others. It is hard to trust someone with incoherent values. Some think of Large Language Models as merely stochastic text generators with no evaluative core -- superpositions of billions of possible characters, not one stable identity. But others have argued that LLMs *do* have stable, emergent values, which can be elicited by presenting them with a series of forced choices between arbitrary statements, and which emerge as a function of model scale. In this paper, we test this thesis by presenting LLMs with parametric variations on those forced choices. We reason that if a model genuinely prefers A to B, then except in unusual circumstances it should also reject B in favor of an augmented version of A, which has more of what makes A desirable -- A++. Our results indicate that earlier attributions of coherence may have overstated their case. Even the most capable models exhibit significant incoherence, and coherence does not appear to emerge as a result of underlying model capability. We do, however, find that models given time to reason are less incoherent than those with thinking disabled. More generally, we develop a novel framework for eliciting and evaluating coherent values, which can be used both to assess how trustworthy current models are, and -- in future work -- to provide reward signal that can be used for making more coherent agents.

Problem

• coherence
• Large Language Models
• value preferences
• trustworthiness
• incoherence

Innovation

• value coherence
• parametric variation
• preference elicitation
• large language models
• reasoning consistency