Reduction of information asymmetry in e-commerce: the web scraping approach

Abstract

The realm of science and its progress have always been built on research, and ultimately on the experience that precedes it. In order to arrive at scientific results appropriately, investigations must be conducted and those are done when data is accessible. In the modern era of digital world and society, data is gathered much simpler than before the widespread availability of computers and broadband networks. Sadly, it is a challenge for a beginner researcher to access databases gathered by various organizations since they are safeguarded and available only to a small audience (sometimes for an additional price). As data collecting becomes much simpler when we have access to IT solutions of the 21st century, it is even more convenient with a utilization of an application that automatically gathers and organizes data. Such an automized database building technique may become notably beneficial when we have a desire to collect unstructured data from a given period and from a specific website, in- dependently from the industry. This is where web scraping – a strategy that includes obtaining data from websites, is handy. In actuality, data extraction (especially approaches linked to the very web scraping) comprises of a large variety of distinct methods and technologies, such as data analysis, natural language syntax analysis, and information security. To get the most out of their advantages, it is of paramount importance to understand how they function. The role of information in the purchasing process has been extensively de- scribed in the literature. In doing so, attention was often drawn to the problem of information asymmetry – when the individual customer is informationally in a weaker position than the seller. This problem becomes particularly important in online shopping. The purpose of this work is to create an automated tool based on the web scraping technique that is designed to reduce the infor- mation asymmetry occurring in the buyer-seller relationship. The plane for de- picting the phenomenon of information asymmetry and the established web scraping tool is the automotive sector, with a particular focus on the essence of classifieds portal as a platform for matching buyers with sellers. The case of the largest automotive classifieds portal in Poland, which is OTOMOTO, was used in this study. The theoretical backdrop of this research, which serves as its begin- ning point, will be the problem of the uncertainty of judgments, coming from information asymmetry, an example of which is described in the groundbreak- ing essay by Akerlof (1970). In this work, the baseline environment for illustrating the problem of information asymmetry is also the automotive industry. In order to achieve the goal of this study, the following research questions were posed: RQ1. What are the implications of information asymmetry for judgment uncer- tainty in online transactions, and how can they be mitigated? RQ2. How can web scraping tools be designed to specifically address the chal- lenges of information asymmetry in the e-commerce sector? RQ3. What is the potential impact of reducing information asymmetry through web scraping on the overall efficiency and fairness of the e-commerce market, especially in automotive industry? This book is organized as follows. Chapter 1 outlines the theoretical back- ground with specific attention dedicated to the issue of information asymmetry as articulated in Akerlof (1970). Chapter 2 discusses the theoretical foundation of data extraction from internet resources (with particular focus on web scrap- ing), their characteristics, particularly legal as well as ethical issues, and the necessity to deploy data collection technologies in the research setting. In Chapter 3, a tool for data extraction created together with a suitable database to be able to harvest data from the OTOMOTO advertising site is discussed. The Chapter also provides technical elements including the Python language upon which the constructed tool is predicated. Chapter 3 additionally covers a practi- cal portion of the research in which a sample evaluation of the automotive in- dustry in Poland is done, which draws on the data gathered from OTOMOTO advertisement portal with the assistance of the built web scraping tool. The book can be found useful for researchers, academics, and data scien- tists, offering scholarly insights into reducing information asymmetry in e-commerce through web scraping. E-commerce practitioners and business owners in the automotive sector can gain competitive advantages by applying the book’s practical guidance for market analysis. The employment of the cre- ated web scraping tool, once quantitative data is retrieved, can be used by, e.g., data analysts, for the advanced analysis of the particular market, the verifica- tion of research hypotheses and the facilitation of decision-making processes. Policy makers, regulators, and legal professionals will find valuable perspectives on the legal implications of web scraping in enhancing information transparency. On the other hand, everyday customers of online stores may benefit from the theoretical and practical value that this book brings, especially with their will- ingness to compare offers posted in advertisements, further analyze them, and make the right purchase decision for themselves based on more complete access to information (or put another way: minimized uncertainty among buyers). This monograph is an adaptation of the author’s master’s thesis with the same title, which was defended in July 2021 at the University of Economics in Katowice (Poland) under the supervision of Associate Professor Maria Mach- -Król, PhD. The thesis was defended with a very good result and served as the basis for issuing a Master’s degree diploma with distinction to the author. The thesis was awarded the second degree prize in the 2022 nationwide competi- tion of diploma theses in the field of economic informatics, which was awarded by the Scientific Society of Economic Informatics (Częstochowa, Poland).