LAMDA: A Longitudinal Android Malware Benchmark for Concept Drift Analysis

2025-05-24

AI Summary

To address persistent model degradation in Android malware detection caused by concept drift, this paper introduces the largest longitudinal benchmark dataset to date—spanning 12 years (2013–2025, excluding 2015), comprising over one million samples and 1,380 malware families, enabling the first high-fidelity cross-decade evolutionary modeling. We propose a systematic evaluation framework integrating temporal slicing, drift quantification (via accuracy decay rate and feature stability), and both static and dynamic features to assess time-series performance of models including Random Forest and XGBoost. Empirical analysis reveals an average annual accuracy decay of 12.7% across mainstream models and identifies highly unstable API and permission features. This dataset and methodology fill a critical long-standing gap in temporal evaluation, providing a robust foundation for research on drift adaptation, incremental learning, and interpretability in evolving Android malware landscapes.

Technology Category

• Machine Learning: Evaluation and Analysis
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Web Mining and Content Analysis: Models for Web evolution
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Machine learning (ML)-based malware detection systems often fail to account for the dynamic nature of real-world training and test data distributions. In practice, these distributions evolve due to frequent changes in the Android ecosystem, adversarial development of new malware families, and the continuous emergence of both benign and malicious applications. Prior studies have shown that such concept drift -- distributional shifts in benign and malicious samples, leads to significant degradation in detection performance over time. Despite the practical importance of this issue, existing datasets are often outdated and limited in temporal scope, diversity of malware families, and sample scale, making them insufficient for the systematic evaluation of concept drift in malware detection. To address this gap, we present LAMDA, the largest and most temporally diverse Android malware benchmark to date, designed specifically for concept drift analysis. LAMDA spans 12 years (2013-2025, excluding 2015), includes over 1 million samples (approximately 37% labeled as malware), and covers 1,380 malware families and 150,000 singleton samples, reflecting the natural distribution and evolution of real-world Android applications. We empirically demonstrate LAMDA's utility by quantifying the performance degradation of standard ML models over time and analyzing feature stability across years. As the most comprehensive Android malware dataset to date, LAMDA enables in-depth research into temporal drift, generalization, explainability, and evolving detection challenges. The dataset and code are available at: https://iqsec-lab.github.io/LAMDA/.

Problem

• ML-based malware detection fails to handle dynamic data distributions
• Existing datasets lack temporal scope and malware diversity for drift analysis
• LAMDA benchmark addresses concept drift in Android malware detection

Innovation

• Largest Android malware benchmark for drift analysis
• 12-year span with 1 million diverse samples
• Quantifies ML model degradation over time