No Vulnerability Data, No Problem: Towards Predicting Mean Time To Remediate In Open Source Software Dependencies

2024-03-26

AI Summary

Assessing timely remediation risk for open-source software dependencies is challenging when vulnerability data are unavailable. Method: This paper introduces MTTR_dep—Mean Time to Remediate for dependencies—and proposes MTTU_dep (Mean Time to Update) as a highly correlated proxy metric, defined as the average interval between successive version releases. MTTU_dep addresses key limitations of existing metrics: neglect of dependency hierarchy, insufficient historical aggregation, and insensitivity to floating-version semantics. Contribution/Results: Through large-scale empirical analysis of 163,000 packages across npm, PyPI, and Cargo ecosystems—combined with version evolution graph modeling and statistical validation—we demonstrate a strong correlation (Spearman ρ > 0.9) between MTTU_dep and MTTR_dep. This enables scalable, low-cost, data-light risk assessment in absence of vulnerability records, advancing quantifiable decision support for software supply chain security.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Timely remediation of vulnerabilities in software dependencies is critical for the security of the software supply chain. As such, researchers have proposed tools and metrics to help practitioners assess the security practices of each of their dependencies. Conceptually, a dependency-focused Mean-Time-To-Remediate (MTTR) metric can provide a historical perspective on how long it takes a given package to update vulnerable versions of its dependencies. However, existing MTTR metrics focus on a package fixing bugs in its own code, not its dependencies. Simultaneously, existing dependency update metrics do not aggregate values for the entire package and are not sensitive to aspects important for vulnerabilities (e.g., floating version constraints). The goal of this study is to aid industry practitioners, including developers, in assessing the risk of dependencies through a novel metric approximating mean-time-to-remediate vulnerabilities in their dependencies that is evaluated by an empirical study. We propose a novel algorithm for computing MTTR called $MTTR_{dep}$ and a companion metric called $Mean-Time-To-Update_{dep}$ ($MTTU_{dep}$), which considers all version updates, including vulnerability fix updates. We conduct a large-scale study using 163, 207 packages in npm, PyPI, and Cargo, of which only 22, 513 packages produce $MTTR_{dep}$ because of the lack of vulnerability data. We further study how package characteristics (e.g., contributors and version counts) influence $MTTU_{dep}$ and $MTTR_{dep}$ and explore how long packages retain outdated vulnerable dependencies in npm, PyPI, and Cargo. Our results indicate that industry practitioners can reliably use $MTTU_{dep}$ as a proxy for $MTTR_{dep}$ when available vulnerability data is insufficient.

Problem

• Predicting mean time to remediate vulnerabilities in open-source software dependencies.
• Developing metrics to assess dependency security practices and risk.
• Evaluating the impact of package characteristics on vulnerability remediation.

Innovation

• Proposes $MTTR_{dep}$ for dependency vulnerability remediation
• Introduces $MTTU_{dep}$ as a proxy for $MTTR_{dep}$
• Analyzes 163,207 packages across npm, PyPI, and Cargo