A Reinforcement Learning Approach to Guide Web Crawler to Explore Web Applications for Improving Code Coverage

Abstract

Web crawlers are widely used to automatically explore and test web applications. However, navigating the pages of a web application can be difficult due to dynamic page generation. In particular, the inputs for the web form fields can affect the resulting pages and subsequent navigation. Therefore, choosing the inputs and the order of clicks on a web page is essential for an effective web crawler to achieve high code coverage. This paper proposes a set of actions to quickly fill in web form fields and uses reinforcement learning algorithms to train a convolutional neural network (CNN). The trained agent, named iRobot, can autonomously select actions to guide the web crawler to maximize code coverage. We experimentally compared different reinforcement learning algorithms, neural networks, and actions. The results show that our CNN network with the proposed actions performs better than other neural networks in terms of branch coverage using the Deep Q-learning (DQN) or proximal policy optimization (PPO) algorithm. Furthermore, compared to previous studies, iRobot can increase branch coverage by about 1.7% while reducing training time to 12.54%.