Multi-LLM Systems Exhibit Robust Semantic Collapse

2026-05-16

AI Summary

This study addresses the long-standing debate over whether large language models (LLMs) possess genuine “originality” by investigating their capacity to sustain novel semantic content generation in closed-loop autonomous interactions. Through simulated experiments involving multi-LLM systems over 200–1,000 interaction rounds, combined with semantic representation analysis and twelve intervention strategies—including decoding parameters, prompt design, agent composition, activation engineering, and reinforcement learning—the work provides the first empirical evidence that irreversible semantic collapse occurs universally across all configurations. Critically, this degradation stems from the intrinsic mechanics of autoregressive generation rather than alignment or conformity biases, revealing a fundamental limitation of multi-LLM systems in open-ended knowledge production.

Abstract

Whether machines can originate novel content has been debated for nearly two centuries, from Lovelace's assertion that no engine can "originate anything" to Turing's question of whether a machine can amplify ideas brought in from outside. Multi-large language model (LLM) systems, increasingly deployed for autonomous generation, reopen this question empirically. Here we show that such systems, operating in closed loops, exhibit semantic collapse: systematic convergence in semantic representations despite apparent lexical variation. Across model families, extended simulations of 200 to 1,000 rounds, the pattern remains consistent. Twelve intervention strategies, spanning decoding parameters, prompt design, agent composition, activation engineering, and reinforcement learning, fail to restore semantic diversity. Mechanistic analyses suggest that semantic collapse is not explained by alignment or conformity biases, but is consistent with intrinsic properties of autoregressive generation. Our results point to fundamental constraints in the ability of multi-LLM systems to sustain open-ended knowledge production in closed-loop settings.

Problem

• semantic collapse
• multi-LLM systems
• autonomous generation
• closed-loop
• open-ended knowledge production

Innovation

• semantic collapse
• multi-LLM systems
• autoregressive generation
• closed-loop generation
• knowledge production