Author:
Gilal Nauman Ullah,Qaraqe Marwa,Schneider Jens,Agus Marco
Abstract
AbstractFood computing has emerged as a promising research field, employing artificial intelligence, deep learning, and data science methodologies to enhance various stages of food production pipelines. To this end, the food computing community has compiled a variety of data sets and developed various deep-learning architectures to perform automatic classification. However, automated food classification presents a significant challenge, particularly when it comes to local and regional cuisines, which are often underrepresented in available public-domain data sets. Nevertheless, obtaining high-quality, well-labeled, and well-balanced real-world labeled images is challenging since manual data curation requires significant human effort and is time-consuming. In contrast, the web has a potentially unlimited source of food data but tapping into this resource has a good chance of corrupted and wrongly labeled images. In addition, the uneven distribution among food categories may lead to data imbalance problems. All these issues make it challenging to create clean data sets for food from web data. To address this issue, we present AutoCleanDeepFood, a novel end-to-end food computing framework for regional gastronomy that contains the following components: (i) a fully automated pre-processing pipeline for custom data sets creation related to specific regional gastronomy, (ii) a transfer learning-based training paradigm to filter out noisy labels through loss ranking, incorporating a Russian Roulette probabilistic approach to mitigate data imbalance problems, and (iii) a method for deploying the resulting model on smartphones for real-time inferences. We assess the performance of our framework on a real-world noisy public domain data set, ETH Food-101, and two novel web-collected datasets, MENA-150 and Pizza-Styles. We demonstrate the filtering capabilities of our proposed method through embedding visualization of the feature space using the t-SNE dimension reduction scheme. Our filtering scheme is efficient and effectively improves accuracy in all cases, boosting performance by 0.96, 0.71, and 1.29% on MENA-150, ETH Food-101, and Pizza-Styles, respectively.
Funder
Hamad bin Khalifa University
Publisher
Springer Science and Business Media LLC
Reference72 articles.
1. Siddiqi, R.: Effectiveness of transfer learning and fine tuning in automated fruit image classification. In: Proceedings of the 2019 3rd International Conference on Deep Learning Technologies, pp. 91–100 (2019)
2. Ciocca, G., Micali, G., Napoletano, P.: State recognition of food images using deep features. IEEE Access 8, 32003–32017 (2020)
3. Gilal, N.U., Al-Thelaya, K., Schneider, J., She, J., Agus, M.: Slowdeepfood: a food computing framework for regional gastronomy. In: Smart Tools and Apps for Graphics—Eurographics Italian Chapter Conference. The Eurographics Association (2021)
4. Subhi, M.A., Ali, S.M.: A deep convolutional neural network for food detection and recognition. In: 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES), pp. 284–287. IEEE (2018)
5. Sun, J., Radecka, K., Zilic, Z.: Exploring better food detection via transfer learning. In: 2019 16th International Conference on Machine Vision Applications (MVA), pp. 1–6. IEEE (2019)