LadyBug: A GitHub Bot for UI-Enhanced Bug Localization in Mobile Apps

2025-08-07

AI Summary

To address low bug localization accuracy in Android applications, this paper proposes a cross-modal bug localization method that jointly models UI interaction traces and natural language bug reports. Specifically, it encodes user-reproduction UI action sequences—including widget IDs and event types—together with textual bug descriptions, leveraging multi-granularity text similarity retrieval and UI behavioral semantic alignment for precise defect localization. The approach is integrated into a GitHub-bot-driven automated testing pipeline. Evaluation on the RedWing benchmark dataset (80 real-world Android bugs) demonstrates significant improvement over text-only retrieval baselines, achieving a 23.6 percentage-point gain in Top-1 accuracy. To foster reproducibility and industrial adoption, the source code and tooling are publicly released under an open-source license.

Technology Category

• Natural Language Processing: Safety and Robustness
• Intelligent Robots: Localization, Mapping, and Navigation
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

This paper introduces LadyBug, a GitHub bot that automatically localizes bugs for Android apps by combining UI interaction information with text retrieval. LadyBug connects to an Android app's GitHub repository, and is triggered when a bug is reported in the corresponding issue tracker. Developers can then record a reproduction trace for the bug on a device or emulator and upload the trace to LadyBug via the GitHub issue tracker. This enables LadyBug to utilize both the text from the original bug description, and UI information from the reproduction trace to accurately retrieve a ranked list of files from the project that most likely contain the reported bug. We empirically evaluated LadyBug using an automated testing pipeline and benchmark called RedWing that contains 80 fully-localized and reproducible bug reports from 39 Android apps. Our results illustrate that LadyBug outperforms text-retrieval-based baselines and that the utilization of UI information leads to a substantial increase in localization accuracy. LadyBug is an open-source tool, available at https://github.com/LadyBugML/ladybug. A video showing the capabilities of Ladybug can be viewed here: https://youtu.be/hI3tzbRK0Cw

Problem

• Automates bug localization in Android apps using UI and text data
• Improves accuracy by combining bug descriptions with UI interaction traces
• Provides ranked list of likely bug-containing files for developers

Innovation

• GitHub bot combines UI and text for bug localization
• Uses reproduction traces from devices or emulators
• Outperforms text-only methods with UI-enhanced accuracy