From Exploration to Specification: LLM-Based Property Generation for Mobile App Testing

2026-04-15

AI Summary

This work addresses the challenge of detecting non-crashing functional defects in mobile applications, which often arise due to the absence of explicit test oracles. To tackle this issue, the paper proposes PropGen, a novel approach that integrates large language models with function-guided exploration to establish a closed-loop pipeline from behavioral observation to precise property generation. PropGen leverages behavioral evidence collection, automated property synthesis, and test-feedback-driven refinement to iteratively improve property accuracy. Evaluated on 12 real-world Android applications, PropGen successfully identified 1,210 valid functionalities, generated 912 effective properties, corrected 118 imprecise ones, and uncovered 25 previously unknown functional defects. These results demonstrate a significant advancement in both the effectiveness and practicality of automated property generation for mobile app testing.

Technology Category

• Natural Language Processing: Generation
• Application Domains: Mobility, Driving & Flight
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Mobile apps often suffer from functional bugs that do not cause crashes but instead manifest as incorrect behaviors under specific user interactions. Such bugs are difficult to detect automatically because they often lack explicit test oracles. Property-based testing can effectively expose them by checking intended behavioral properties under diverse interactions. However, its use largely depends on manually written properties, whose construction is difficult and expensive, limiting its practical use for mobile apps. To address this limitation, we propose PropGen, an automated approach for generating properties for Android apps. However, this task is challenging for two reasons: app functionalities are often hard to systematically uncover and execute, and properties are difficult to derive accurately from observed behaviors. To this end, PropGen performs functionality-guided exploration to collect behavioral evidence from app executions, synthesizes properties from the collected evidence, and refines imprecise properties based on testing feedback. We implemented PropGen and evaluated it on 12 real-world Android apps. The results show that PropGen can effectively identify and execute valid app functionalities, generate valid properties, and repair most imprecise ones. Across all apps, PropGen identified 1,210 valid functionalities and correctly executed 977 of them, compared with 491 and 187 for the baseline. It generated 985 properties, 912 of which were valid, and repaired 118 of 127 imprecise ones exposed during testing. With the resulting properties, we found 25 previously unknown functional bugs in the latest versions of the subject apps, many of which were missed by existing functional testing techniques.

Problem

• functional bugs
• test oracles
• property-based testing
• mobile app testing
• behavioral properties

Innovation

• property-based testing
• LLM-based property generation
• functionality-guided exploration
• automated test oracle synthesis
• mobile app testing