The Emerging Threats of Web Scrapping to Web Applications Security and Their Defense Mechanism

Abstract

Web scraping is the technique exploited to robotically obtain particular information from web applications instead of manually copying it. The purpose of a web scraper is to search for certain class of information, dig out, and aggregate it into new database. More precisely, web scrapers are used to transform unstructured web data and store them in structured databases. It is a continuing threat to web applications that aims to steal sensitive data from a victim or from web applications. The key objective of this article is to examine to what extent web scraping can cause a threat to web application security. This article explores the classification of web scraping such as content scraping, web scraping, price scraping, and database scraping in general and presents the most widely used scraping tools such as Web Content Extractor, and Screen Scrapper. Consequently, the aim of this article is to give evaluation of vulnerabilities, threats of web scraping associated with web application applications, and effective measures to counter them.