Design and Implementation of a Recommendation System for Buying Fresh Foods Online Based on Web Crawling

Abstract

As shopping patterns have gradually shifted from offline to online mode, and with recent lockdowns during the coronavirus disease 2019 (COVID-19) pandemic restricting foreign trade and accelerating the growth of the domestic economy, digital transformation has become a major strategy for many retailers to support and expand their businesses. With the pandemic becoming a turning point, the business of major e-commerce companies in Taiwan in the retail of dry goods has grown significantly, and it has driven the online sales of fresh products as well. In this era of fierce competition, it is especially important to find a way that enables consumers to quickly find ideal fresh products on multiple platforms, shortens the time for price comparison, and improves the efficiency of online shopping. This study uses the Python programming language to write a web crawler program that captures product information from fresh food e-commerce platforms, including product introduction, price, origin, and sales volume, and then defines the relevant status of the product, such as product popularity. Accordingly, through Chinese text segmentation and term-frequency calculation, it aims to classify the product names and introductions into frequently occurring words and use them as product-related labels. Finally, the program combines the product information processing results and product-related labels to construct an online fresh food recommendation system. The results of the proposed system show that it reduces the time and energy spent comparing prices. It can also guide consumers to browse products that may be of interest using relevant tags and increase consumption efficiency by helping them find the ideal item when shopping.