A new membership inference attack that spots memorization in generative and predictive models: Loss-Based with Reference Model algorithm (LBRM)

2025-05-06

AI Summary

Time-series imputation models often inadvertently memorize training data during training, posing privacy risks such as membership inference attacks (MIAs). To address this, we propose Loss-and-Reference-based Membership inference (LBRM), a novel framework that detects model memorization by comparing the loss distribution of a target imputation model against that of a lightweight reference model—without requiring fine-tuning. LBRM is the first dedicated memory-detection framework for time-series imputation models and is systematically validated across GAN- and VAE-based architectures. It achieves an AUROC improvement of approximately 40% without fine-tuning and up to 60% with fine-tuning, demonstrating strong robustness and cross-architecture generalizability. This work is the first to empirically reveal and quantify data memorization in time-series imputation models, establishing a new privacy evaluation benchmark specifically tailored for imputation tasks.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Adversarial Attacks & Robustness
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Generative models can unintentionally memorize training data, posing significant privacy risks. This paper addresses the memorization phenomenon in time series imputation models, introducing the Loss-Based with Reference Model (LBRM) algorithm. The LBRM method leverages a reference model to enhance the accuracy of membership inference attacks, distinguishing between training and test data. Our contributions are twofold: first, we propose an innovative method to effectively extract and identify memorized training data, significantly improving detection accuracy. On average, without fine-tuning, the AUROC improved by approximately 40%. With fine-tuning, the AUROC increased by approximately 60%. Second, we validate our approach through membership inference attacks on two types of architectures designed for time series imputation, demonstrating the robustness and versatility of the LBRM approach in different contexts. These results highlight the significant enhancement in detection accuracy provided by the LBRM approach, addressing privacy risks in time series imputation models.

Problem

• Detects memorization in generative and predictive models
• Improves membership inference attack accuracy using LBRM
• Addresses privacy risks in time series imputation models

Innovation

• LBRM algorithm enhances membership inference attack accuracy
• Uses reference model to distinguish training and test data
• Improves AUROC by 40-60% in detection accuracy