Few-Shot Class-Incremental Model Attribution Using Learnable Representation From CLIP-ViT Features

2025-03-11

AI Summary

This work addresses the model attribution (MA) challenge arising from the continual emergence of generative AI models. We propose a few-shot class-incremental learning (FSCIL) framework that enables accurate identification of novel generative models without retraining. To our knowledge, this is the first application of FSCIL to MA. Our method introduces an Adaptive Integration Module (AIM) that fuses multi-level features from CLIP-ViT to jointly capture low-level texture and high-level semantic cues; a learnable weighted fusion strategy dynamically aggregates discriminative cross-layer features. Evaluated across generational generative models—including GANs, diffusion models, and LLM-based generators—our approach achieves high attribution accuracy using only 1–5 samples per class. It significantly reduces both data requirements and computational overhead, consistently outperforming state-of-the-art MA methods in comprehensive experiments.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recently, images that distort or fabricate facts using generative models have become a social concern. To cope with continuous evolution of generative artificial intelligence (AI) models, model attribution (MA) is necessary beyond just detection of synthetic images. However, current deep learning-based MA methods must be trained from scratch with new data to recognize unseen models, which is time-consuming and data-intensive. This work proposes a new strategy to deal with persistently emerging generative models. We adapt few-shot class-incremental learning (FSCIL) mechanisms for MA problem to uncover novel generative AI models. Unlike existing FSCIL approaches that focus on object classification using high-level information, MA requires analyzing low-level details like color and texture in synthetic images. Thus, we utilize a learnable representation from different levels of CLIP-ViT features. To learn an effective representation, we propose Adaptive Integration Module (AIM) to calculate a weighted sum of CLIP-ViT block features for each image, enhancing the ability to identify generative models. Extensive experiments show our method effectively extends from prior generative models to recent ones.

Problem

• Addresses model attribution for evolving generative AI models
• Uses few-shot class-incremental learning for recognizing new models
• Analyzes low-level image details to identify generative models

Innovation

• Few-shot class-incremental learning for model attribution
• Learnable representation from CLIP-ViT features
• Adaptive Integration Module for feature weighting