VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Models

2026-02-03

AI Summary

This work addresses the challenge of aligning large language models with diverse human values, a task hindered by limitations in existing approaches regarding value extraction, intensity quantification, and controllable guidance. To overcome these issues, the authors propose VALUEFLOW, a unified framework that integrates, for the first time, a hierarchical value embedding space (HIVES), a value-annotated corpus with intensity labels (VIDB), and an anchor-based pairwise ranking evaluation mechanism. This integration enables fine-grained, quantifiable, and controllable value alignment. Experiments across ten models and four value theories reveal asymmetric effects and compositional patterns in value steering, while establishing a scalable infrastructure for precise control over value intensity.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Philosophy and Ethics of AI: Morality & Value-based AI
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Aligning Large Language Models (LLMs) with the diverse spectrum of human values remains a central challenge: preference-based methods often fail to capture deeper motivational principles. Value-based approaches offer a more principled path, yet three gaps persist: extraction often ignores hierarchical structure, evaluation detects presence but not calibrated intensity, and the steerability of LLMs at controlled intensities remains insufficiently understood. To address these limitations, we introduce VALUEFLOW, the first unified framework that spans extraction, evaluation, and steering with calibrated intensity control. The framework integrates three components: (i) HIVES, a hierarchical value embedding space that captures intra- and cross-theory value structure; (ii) the Value Intensity DataBase (VIDB), a large-scale resource of value-labeled texts with intensity estimates derived from ranking-based aggregation; and (iii) an anchor-based evaluator that produces consistent intensity scores for model outputs by ranking them against VIDB panels. Using VALUEFLOW, we conduct a comprehensive large-scale study across ten models and four value theories, identifying asymmetries in steerability and composition laws for multi-value control. This paper establishes a scalable infrastructure for evaluating and controlling value intensity, advancing pluralistic alignment of LLMs.

Problem

• value alignment
• large language models
• value intensity
• pluralistic alignment
• steerability

Innovation

• value-based alignment
• hierarchical embedding
• intensity calibration
• steerable LLMs
• pluralistic values