Reactive Bottom-Up Testing

2025-09-03

AI Summary

Modern software often contains deep-function vulnerabilities that evade detection by conventional top-down dynamic testing. To address this, we propose a reactive bottom-up testing paradigm. Our approach introduces the first three-stage closed-loop framework: (1) generating isolated test harnesses guided by type and contextual awareness; (2) extracting and composing input constraints via hybrid fuzzing and symbolic execution; and (3) validating vulnerability reachability and exploitability within the full program context. Unlike prior bottom-up methods, our technique substantially reduces false positives while ensuring detected vulnerabilities are genuinely exploitable. Evaluation on a benchmark of 48 known vulnerabilities achieves a detection rate of 58.3% (28/48). Furthermore, our method discovers six previously unknown vulnerabilities in real-world systems—including Pacman—demonstrating significantly enhanced capability for detecting deep-seated vulnerabilities in complex software.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Security and Privacy: Large-scale security measurements
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Modern computing systems remain rife with software vulnerabilities. Engineers apply many means to detect them, of which dynamic testing is one of the most common and effective. However, most dynamic testing techniques follow a top-down paradigm, and struggle to reach and exercise functions deep within the call graph. While recent works have proposed Bottom-Up approaches to address these limitations, they face challenges with false positives and generating valid inputs that adhere to the context of the entire program. In this work, we introduce a new paradigm that we call Reactive Bottom-Up Testing. Our insight is that function-level testing is necessary but not sufficient for the validation of vulnerabilities in functions. What we need is a systematic approach that not only tests functions in isolation but also validates their behavior within the broader program context, ensuring that detected vulnerabilities are both reachable and triggerable. We develop a three-stage bottom-up testing scheme: (1) identify likely-vulnerable functions and generate type- and context-aware harnesses; (2) fuzz to find crashes and extract input constraints via symbolic execution; (3) verify crashes by combining constraints to remove false positives. We implemented an automated prototype, which we call Griller. We evaluated Griller in a controlled setting using a benchmark of 48 known vulnerabilities across 5 open-source projects, where we successfully detected 28 known vulnerabilities. Additionally, we evaluated Griller on several real-world applications such as Pacman, and it discovered 6 previously unknown vulnerabilities. Our findings suggest that Reactive Bottom-Up Testing can significantly enhance the detection of vulnerabilities in complex systems, paving the way for more robust security practices.

Problem

• Detecting deep software vulnerabilities in complex systems
• Reducing false positives in bottom-up dynamic testing approaches
• Generating context-aware inputs for program function validation

Innovation

• Reactive Bottom-Up Testing paradigm
• Three-stage function validation scheme
• Automated harness generation and constraint extraction