Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models

2026-08-11

AI Summary

This study investigates the origins of output homogenization in large language models, challenging the prevailing view that it primarily stems from alignment processes. Through controlled instruction fine-tuning experiments, prompt-induced tests, and cross-model consistency measurements, the authors systematically analyze behavioral differences between base and aligned models. They find that semantic convergence is already inherent in models during pretraining and is merely revealed or amplified during alignment; instruction fine-tuning does not introduce homogenization. Remarkably, base models—without any alignment—can exhibit alignment-like output convergence when prompted appropriately. This work provides the first evidence that homogenization may originate from the pretraining objective itself.

Abstract

The lack of diversity in LM content is widely attributed to the alignment process, but how and where exactly in the pipeline this collapse begins is unknown. We argue that output homogeneity is likely learned during the pretraining phase, and only \emph{revealed} or magnified during the alignment process. Specifically, we find that semantic convergence is observed from the first alignment stage--the instruction-tuning phase (SFT)--suggesting that homogeneity might already exist in the pre-alignment model. To investigate this, we conduct controlled SFT experiments examining how training data influences output convergence on specific input/output pairs. We find that convergence can be revealed and amplified, but not introduced by the SFT data, supporting its role as a catalyst rather than a cause. To further test whether homogeneity originates before alignment, we measure convergence in base models. We find that instruct-like collapse can be induced through prompting alone, even without alignment. Taken together, our results suggest that semantic convergence may arise naturally from the objectives underlying LM training, making it difficult to mitigate through post-alignment interventions alone.

Problem

• output homogeneity
• semantic convergence
• pretraining
• alignment
• large language models

Innovation

• output homogeneity
• pretraining
• instruction tuning
• semantic convergence
• base models