Converge to Surprise: Evolutionary Self-supervised Image Clustering

2026-07-07

AI Summary

This work addresses the challenges in unsupervised image clustering where the number of categories is unknown and differentiable loss functions are lacking. It proposes a non-parametric, self-supervised clustering framework that eschews explicit objectives and instead constructs a null hypothesis grounded in the principle of maximum entropy. The method employs a non-differentiable “surprise score” as its optimization target to guide the model toward discovering non-random structures in the data. A novel bi-level optimization mechanism—dubbed “converge-to-surprise”—is introduced, integrating evolutionary strategies with periodic gradient descent for efficient training. Evaluated on standard image benchmarks, the approach achieves state-of-the-art performance in non-parametric self-supervised clustering without requiring any prior specification of the number of clusters.

Abstract

Most self-supervised image clustering models, actually almost all deep learning approaches, are based on gradient descent: In order to calculate the loss, every optimization step requires a clearly defined target, whether a contrastive split, a masked patch or entity, an EMA-teacher output, a pseudo-label, or a differentiable information-theoretic functional. We propose a self-supervised framework that drops this requirement for image clustering. Without any prior knowledge, we have to assume that each pixel is i.i.d. according to the Principle of Maximum Entropy. Taking this as our null hypothesis H0, we define a "surprise score" that measures how unlikely the model's output representation would be under H0. Maximizing the surprise score forces the deep learning model to reject H0 - equivalently, to discover non-random feature from data. Also, here is our fundamental assumption: a surprise score cannot, in general, be reduced to a per-step loss. Hence, we propose the "converge-to-surprise" scheme to optimize our model: an evolution-strategy (ES) outer loop, which directly maximizes the surprise score without needing its gradient, paired with a periodic gradient-descent inner loop, which uses the surprising clusters already discovered by ES as surrogate targets. On standard image benchmarks, our framework achieves new state-of-the-art results in non-parametric self-supervised image clustering - the strictest deep-clustering setting, in which the number of ground-truth classes is not given to the model.

Problem

• self-supervised image clustering
• gradient descent
• surprise score
• maximum entropy
• non-parametric clustering

Innovation

• surprise score
• evolution strategy
• self-supervised clustering
• maximum entropy
• gradient-free optimization