Visual judgment approach of isomorphic GUI for automated mobile app testing

Abstract

Currently, the rapid growth of mobile apps requires automated testing technology to ensure their quality. Automated testing of mobile apps is usually closely related to the recognition and judgment of their graphical user interface (GUI), but there usually are numerous isomorphic GUIs with different styles and contents, and similar structure and function in mobile apps. In automatic testing, isomorphic GUI is easy to cause the issue of state space explosion, which leads to low efficiency or failure of testing. In view of the limitations of traditional automatic recognition of isomorphic GUI, this paper presents a GUI similarity judgment approach based on visual feature information. Firstly, the GUI component elements are identified by object detection, and then the GUI skeleton is abstracted. Secondly, the visual features of the GUI skeleton are extracted by a convolutional autoencoder. Finally, the isomorphic GUI judgment is completed by comparing the similarity of GUI visual features. The experimental results show that the proposed approach can effectively shield the influence of GUI style and content, complete the isomorphic GUI recognition more accurately and optimize the efficiency of automated mobile app testing.