Automated photo filtering for tourism domain using deep and active learning: the case of Israeli and worldwide cities on instagram

Abstract

AbstractSocial media platforms like Instagram significantly influence tourists’ travel decisions by providing them with valuable insights, recommendations, authentic information, and points of interest. However, photos shared with location-specific hashtags, even those related to tourist attractions, do not always reflect the actual destination, creating challenges for potential visitors seeking accurate information. To assist tourists in finding pertinent tourism information for specific destinations, we propose VISTA: Visual Identification of Significant Travel Attractions. The proposed method employs deep learning and active learning techniques to automatically classify photos into: ‘Tourism-Related’ photos (i.e., photos related to tourism) and ‘Non-Tourism-Related’ photos (i.e., photos unrelated to tourism). To train our machine learning classifier, we created a dataset containing photos of the 10 most popular Israeli cities on Instagram. The classifier obtained an accuracy score of 0.965 and a weighted F1 score of 0.964. Evaluating our classifier’s global generalization on the InstaCities100K dataset, derived from InstaCities1M, yielded an accuracy score of 0.958 and a weighted F1 score of 0.959. The effectiveness of VISTA was demonstrated by comparing tourism-related and non-tourism-related photos in terms of photo proportion, user engagement, and object comparison. We found that most photos published on Instagram associated with cities are irrelevant to tourists and that tourism-related photos received more likes than non-tourism-related photos. Finally, there was a low overlap between objects in the two photo collections. Based on these results, we conclude that VISTA can help tourists tackle the problem of finding relevant tourism-related photos among the high volume of photos available on Instagram.