Intelligence at the Edge of Chaos

2024-10-03

arXiv.org

AI Summary

This work investigates how the complexity of rule-based systems drives the emergence of intelligent behavior. Using elementary cellular automata (ECA) as a controlled experimental testbed, we systematically modulate dynamical complexity—spanning homogeneous, periodic, and chaotic regimes—and train large language models (LLMs) on synthetic data generated by diverse ECA rules. We then evaluate LLM generalization performance on downstream tasks including logical reasoning and chess move prediction. Contrary to the intuitive “more complexity, better intelligence” hypothesis, we identify a pronounced *complexity sweet spot*: ECA rules with intermediate chaos level yield significantly superior model performance—outperforming both low-complexity (homogeneous/periodic) and high-complexity (strongly chaotic) regimes. This finding provides the first empirical evidence of a nonlinear, nonmonotonic relationship between system complexity and emergent intelligence. It establishes a novel constructive paradigm for engineering intelligence in artificial systems, demonstrating that optimal complexity—not maximal—is critical for inducing cognitive capabilities in learned models.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Multiagent Systems: Agent-Based Simulation and Emergent Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

We explore the emergence of intelligent behavior in artificial systems by investigating how the complexity of rule-based systems influences the capabilities of models trained to predict these rules. Our study focuses on elementary cellular automata (ECA), simple yet powerful one-dimensional systems that generate behaviors ranging from trivial to highly complex. By training distinct Large Language Models (LLMs) on different ECAs, we evaluated the relationship between the complexity of the rules' behavior and the intelligence exhibited by the LLMs, as reflected in their performance on downstream tasks. Our findings reveal that rules with higher complexity lead to models exhibiting greater intelligence, as demonstrated by their performance on reasoning and chess move prediction tasks. Both uniform and periodic systems, and often also highly chaotic systems, resulted in poorer downstream performance, highlighting a sweet spot of complexity conducive to intelligence. We conjecture that intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity.

Problem

• Exploring how rule complexity affects AI intelligence.
• Investigating intelligence emergence in cellular automata-trained LLMs.
• Identifying complexity sweet spot for optimal AI performance.

Innovation

• Explored intelligence in rule-based systems
• Used Large Language Models on cellular automata
• Identified complexity sweet spot for intelligence