Affiliation:
1. East China Normal University, China
2. Nanjing University, China
3. Visa Research, USA
4. ETH Zurich, Switzerland
Abstract
Android apps are GUI-based event-driven software and have become ubiquitous in recent years. Obviously, functional correctness is critical for an app’s success. However, in addition to crash bugs,
non-crashing functional bugs
(in short as “non-crashing bugs” in this work) like inadvertent function failures, silent user data lost and incorrect display information are prevalent, even in popular, well-tested apps. These non-crashing functional bugs are usually caused by program logic errors and manifest themselves on the graphic user interfaces (GUIs). In practice, such bugs pose significant challenges in effectively detecting them because (1) current practices heavily rely on expensive, small-scale manual validation (
the lack of automation
); and (2) modern
fully automated
testing has been limited to crash bugs (
the lack of test oracles
).
This paper fills this gap by introducing
independent view fuzzing
,
a novel, fully automated approach
for detecting non-crashing functional bugs in Android apps. Inspired by metamorphic testing, our key insight is to leverage the commonly-held
independent view property
of Android apps to manufacture property-preserving mutant tests from a set of seed tests that validate certain app properties. The mutated tests help exercise the tested apps under additional, adverse conditions. Any property violations indicate likely functional bugs for further manual confirmation. We have realized our approach as an automated, end-to-end functional fuzzing tool, Genie. Given an app, (1) Genie automatically detects non-crashing bugs without requiring human-provided tests and oracles (thus
fully automated
); and (2) the detected non-crashing bugs are diverse (thus
general and not limited to specific functional properties
), which set Genie apart from prior work.
We have evaluated Genie on 12 real-world Android apps and successfully uncovered 34 previously unknown non-crashing bugs in their latest releases — all have been confirmed, and 22 have already been fixed. Most of the detected bugs are nontrivial and have escaped developer (and user) testing for at least one year and affected many app releases, thus clearly demonstrating Genie’s effectiveness. According to our analysis, Genie achieves a reasonable true positive rate of 40.9%, while these 34 non-crashing bugs could not be detected by prior fully automated GUI testing tools (as our evaluation confirms). Thus, our work complements and enhances existing manual testing and fully automated testing for crash bugs.
Publisher
Association for Computing Machinery (ACM)
Subject
Safety, Risk, Reliability and Quality,Software
Reference98 articles.
1. Accessibility. 2020. Accessibility overview. https://developer.android.com/guide/topics/ui/accessibility Accessibility. 2020. Accessibility overview. https://developer.android.com/guide/topics/ui/accessibility
2. ActivityDiary. 2020. Activity Diary. https://play.google.com/store/apps/details?id=de.rampro.activitydiary ActivityDiary. 2020. Activity Diary. https://play.google.com/store/apps/details?id=de.rampro.activitydiary
3. ActivityDiary. 2020. Activity Diary’s issue #118. https://github.com/ramack/ActivityDiary/issues/118 ActivityDiary. 2020. Activity Diary’s issue #118. https://github.com/ramack/ActivityDiary/issues/118
4. Systematic execution of Android test suites in adverse conditions
5. ADB. 2020. Android adb. https://developer.android.com/studio/command-line/adb ADB. 2020. Android adb. https://developer.android.com/studio/command-line/adb
Cited by
19 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Understanding the Reproducibility Issues of Monkey for GUI Testing;Dependable Software Engineering. Theories, Tools, and Applications;2023-12-15
2. Automata-Based Trace Analysis for Aiding Diagnosing GUI Testing Tools for Android;Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering;2023-11-30
3. Property-Based Fuzzing for Finding Data Manipulation Errors in Android Apps;Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering;2023-11-30
4. An Empirical Study of Functional Bugs in Android Apps;Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis;2023-07-12
5. DDLDroid: Efficiently Detecting Data Loss Issues in Android Apps;Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis;2023-07-12