Preventing Model Collapse via Contraction-Conditioned Neural Filters

2025-11-30

AI Summary

Recursive training of generative models suffers from model collapse due to error accumulation. Method: This paper proposes a neural network filtering approach based on contraction operators, embedding contraction conditions directly into an end-to-end learning framework. It designs a dedicated network architecture and customized loss function leveraging exponential-family distribution properties to learn theoretically guaranteed contractive filters under fixed sample size. Contribution/Results: The method ensures almost-sure convergence of estimation error in probability without requiring increased sample complexity—eliminating the traditional reliance on superlinear sample growth. We prove the equivalence between the contraction condition and error convergence. Experiments demonstrate significantly improved long-term training stability for generative models and effective prevention of performance degradation.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper presents a neural network filter method based on contraction operators to address model collapse in recursive training of generative models. Unlike cite{xu2024probabilistic}, which requires superlinear sample growth ($O(t^{1+s})$), our approach completely eliminates the dependence on increasing sample sizes within an unbiased estimation framework by designing a neural filter that learns to satisfy contraction conditions. We develop specialized neural network architectures and loss functions that enable the filter to actively learn contraction conditions satisfying Assumption 2.3 in exponential family distributions, thereby ensuring practical application of our theoretical results. Theoretical analysis demonstrates that when the learned contraction conditions are satisfied, estimation errors converge probabilistically even with constant sample sizes, i.e., $limsup_{t oinfty}mathbb{P}(|mathbf{e}_t|>δ)=0$ for any $δ>0$. Experimental results show that our neural network filter effectively learns contraction conditions and prevents model collapse under fixed sample size settings, providing an end-to-end solution for practical applications.

Problem

• Prevents model collapse in generative model recursive training
• Eliminates dependence on increasing sample sizes for unbiased estimation
• Ensures error convergence with constant sample sizes via neural filters

Innovation

• Neural filter learns contraction conditions to prevent collapse
• Eliminates need for increasing sample sizes in training
• Ensures error convergence with constant sample sizes