ForTIFAI: Fending Off Recursive Training Induced Failure for AI Models

2025-09-10

AI Summary

Generative AI has produced vast amounts of synthetic data; however, recursive training on self-generated data leads to model collapse—progressive degradation of performance across generations. This work identifies excessive model confidence on synthetic data as the primary cause of collapse and proposes a model-agnostic Truncated Cross-Entropy (TCE) loss function. TCE incorporates a confidence-aware mechanism that down-weights high-confidence predictions, thereby mitigating overfitting at the optimization objective level. Theoretical analysis and cross-modal experiments demonstrate that TCE extends the fidelity-preserving training interval before collapse by over 2.3×, significantly enhancing model robustness and stability under synthetic-data training. To our knowledge, TCE constitutes the first general-purpose, provably effective framework for safely leveraging synthetic data—offering formal guarantees against collapse while remaining architecture- and modality-agnostic.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The increasing reliance on generative AI models has accelerated the generation rate of synthetic data, with some projections suggesting that most available new data for training could be machine-generated by 2030. This shift to a mainly synthetic content presents a critical challenge: repeated training in synthetic data leads to a phenomenon known as model collapse, where model performance degrades over generations of training, eventually rendering the models ineffective. Although prior studies have explored the causes and detection of model collapse, existing mitigation strategies remain limited. In this paper, we identify model overconfidence in their self-generated data as a key driver of collapse. Building on this observation, we propose a confidence-aware loss function that downweights high-confidence predictions during training. We introduce a novel loss function we call Truncated Cross Entropy (TCE). We demonstrate that TCE significantly delays model collapse in recursive training. We provide a model-agnostic framework that links the loss function design to model collapse mitigation and validate our approach both theoretically and empirically, showing that it can extend the model's fidelity interval before collapse by more than 2.3x. Finally, we show that our method generalizes across modalities. These findings suggest that the design of loss functions provides a simple yet powerful tool for preserving the quality of generative models in the era of increasing synthetic data.

Problem

• Addresses model collapse from recursive training on synthetic data
• Mitigates performance degradation in generative AI models
• Proposes confidence-aware loss to delay training-induced failure

Innovation

• Confidence-aware loss function design
• Truncated Cross Entropy implementation
• Model-agnostic collapse mitigation framework