It's Much Easier for Neural Networks to learn Game of Life Dynamics with the Right Activation Function: Polynomial Kolmogorov-Arnold Networks

2026-06-22

AI Summary

This work addresses the challenge that neural networks typically require large architectures and specialized initialization to learn the rules of Conway’s Game of Life, often failing to converge reliably in small models. The authors propose incorporating task-specific inductive biases to transform this difficult search problem into a tractable learning task. Their key innovation is a compact network architecture based on the Kolmogorov–Arnold representation theorem, employing a second-order polynomial activation function. Experimental results demonstrate that this design consistently reproduces the dynamics of the Game of Life—both with and without weight training—and substantially outperforms standard activation functions such as ReLU. These findings challenge the prevailing paradigm that model scale is the primary determinant of performance in such rule-learning tasks.

Abstract

Previous work has found a gap between the scale of neural networks that reliably learn Conway's Game of Life, and minimal networks capable of representing the classic cellular automaton with hard-coded parameter values. Viewing neural network learning as a search process suggests a dependence on networks large enough to contain sub-networks with lucky initializations (sometimes known as 'winning tickets') that actually learn the task. In this work, we reorient our perspective from discovering Life rules as a search problem back to a learning problem, and reason that with fitting inductive biases, the problem should be much more amenable to minimal networks. We find that network variants with several alternative activation functions meaningfully outperform the default choice of Rectified Linear Units, and in particular, that a 2nd degree polynomial activation function consistently learns Life dynamics with or without the benefit of learning neural weights. Our results provide an informative demonstration of the benefits of matching learning to the task at hand and challenge the easy default choice of scale for all problems. In particular, we advocate for the use of cellular automata as simple test domains for developing strategies that can benefit machine learning for science, physics-based deep learning, and interpretable machine learning.

Problem

• Conway's Game of Life
• neural network learning
• inductive bias
• activation function
• cellular automata

Innovation

• polynomial activation
• inductive bias
• cellular automata
• minimal networks
• Kolmogorov-Arnold Networks