Distribution-Free Pretraining of Classification Losses via Evolutionary Dynamics

2026-05-05

AI Summary

This work proposes the Evolutionary Dynamic Loss (EDL) framework, a novel approach to pretraining transferable classification loss functions without access to the true data distribution. EDL constructs an infinite set of synthetic prediction–label pairs in probability space and optimizes a lightweight neural network–parameterized loss function via a semantics-agnostic ranking consistency objective. To enhance exploration under noisy fitness evaluations, the method incorporates chaotic mutation into its evolutionary strategy. As the first distribution-agnostic, real-data-free loss pretraining scheme, EDL serves as a plug-and-play replacement for cross-entropy. Experiments on CIFAR-10 with a ResNet backbone demonstrate that EDL achieves comparable or superior accuracy, while chaotic mutation significantly accelerates convergence and improves pretraining performance.

Abstract

We propose Evolutionary Dynamic Loss (EDL), a framework that learns a transferable classification loss in the probability space using unlimited synthetic prediction-label pairs, without accessing real samples during the main loss pretraining stage. EDL parameterizes the loss as a lightweight network and is trained with a semantics-free ranking-consistency objective that assigns larger penalties for more erroneous predictions. To robustly explore the space of loss functions, we optimize EDL via an evolutionary strategy and introduce chaotic mutation to improve exploration under noisy fitness evaluations. Experiments on CIFAR-10 with ResNet backbones show that EDL can serve as a drop-in replacement for cross-entropy and achieves competitive or improved accuracy, while ablation studies confirm that chaotic mutation yields faster convergence and better synthetic pretraining metrics than standard Gaussian mutation.

Problem

• distribution-free pretraining
• classification loss
• synthetic data
• transferable loss
• evolutionary dynamics

Innovation

• Evolutionary Dynamic Loss
• distribution-free pretraining
• chaotic mutation
• synthetic prediction-label pairs
• ranking-consistency objective