Revisiting Code Debloating with Ground Truth-based Evaluation

2026-04-19

AI Summary

Existing code debloating research relies on proxy metrics such as test coverage or code size, lacking evaluation grounded in actual program behavior. This work presents the first unified evaluation framework benchmarked against real-world execution semantics, integrating dynamic and static analysis to systematically reassess eight state-of-the-art debloating tools across source code, intermediate representations, and binary levels. The study reveals that dynamic approaches can erroneously remove up to 94% of code that should be retained, while static methods suffer from high false retention rates due to over-approximation—sometimes even introducing new, unintended function variants—that critically compromise program correctness and security. These findings expose systematic biases in current debloating techniques and establish an empirical foundation for future tool development.

Technology Category

• Machine Learning: Evaluation and Analysis
• Search and Optimization: Evaluation and Analysis
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

Program debloating aims to remove unused code to reduce performance overhead, attack surfaces, and maintenance costs. Over time, debloating has evolved across multiple layers (container, library, and application), each building on the principles of application-level debloating. Despite its central role, application-level debloating continues to rely on imperfect proxies for measuring performance, such as test-case-driven evaluation for correctness, code size for runtime efficiency, and gadget count reduction for estimating security posture. While there is widespread skepticism about using such imperfect proxies, the community still lacks standardized methodologies or benchmarks to assess the true performance of application-level software debloating. This experience paper aims to address the gap. We revisit the foundations of application-level debloating through a ground-truth-based evaluation paradigm. Our analysis of eight state-of-the-art debloaters - Blade, Chisel, Cov, CovA, Lmcas, Trimmer, Occam, and Razor - uncovers insights previously unattainable through traditional evaluations. These tools collectively span the spectrum of source-to-source, IR-to-IR, and binary-to-binary transformation paradigms, characterizing a holistic reassessment across abstraction levels. Our analysis reveals that while dynamic analysis-based tools often remove up to 94% of code that should be retained, static analysis-based approaches exhibit the opposite behavior, showing high false retention rates due to coarse-grained dependency over-approximation. Additionally, static analyses may add code by introducing specialized variants of functions. False retentions and removals not only cause functional incorrectness but may also lead to systematic inconsistency, robustness failures, and exploitable vulnerabilities.

Problem

• code debloating
• ground truth
• evaluation methodology
• application-level
• performance assessment

Innovation

• ground-truth evaluation
• code debloating
• static analysis
• dynamic analysis
• software security