Food Image Segmentation Using Multi-Modal Imaging Sensors with Color and Thermal Data
Author:
Raju Viprav B., Imtiaz Masudul H., Sazonov EdwardORCID
Abstract
Sensor-based food intake monitoring has become one of the fastest-growing fields in dietary assessment. Researchers are exploring imaging-sensor-based food detection, food recognition, and food portion size estimation. A major problem that is still being tackled in this field is the segmentation of regions of food when multiple food items are present, mainly when similar-looking foods (similar in color and/or texture) are present. Food image segmentation is a relatively under-explored area compared with other fields. This paper proposes a novel approach to food imaging consisting of two imaging sensors: color (Red–Green–Blue) and thermal. Furthermore, we propose a multi-modal four-Dimensional (RGB-T) image segmentation using a k-means clustering algorithm to segment regions of similar-looking food items in multiple combinations of hot, cold, and warm (at room temperature) foods. Six food combinations of two food items each were used to capture RGB and thermal image data. RGB and thermal data were superimposed to form a combined RGB-T image and three sets of data (RGB, thermal, and RGB-T) were tested. A bootstrapped optimization of within-cluster sum of squares (WSS) was employed to determine the optimal number of clusters for each case. The combined RGB-T data achieved better results compared with RGB and thermal data, used individually. The mean ± standard deviation (std. dev.) of the F1 score for RGB-T data was 0.87 ± 0.1 compared with 0.66 ± 0.13 and 0.64 ± 0.39, for RGB and Thermal data, respectively.
Funder
National Institute of Diabetes and Digestive Bill & Melinda Gates Foundation
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference53 articles.
1. Zhu, F., Bosch, M., Boushey, C.J., and Delp, E.J. (2010, January 26–29). An Image Analysis System for Dietary Assessment and Evaluation. Proceedings of the 2010 IEEE International Conference on Image Processing, Hong Kong, China. 2. Jia, W., Li, Y., Qu, R., Baranowski, T., Burke, L.E., Zhang, H., Bai, Y., Mancino, J.M., Xu, G., and Mao, Z.-H. (2018). Automatic Food Detection in Egocentric Images Using Artificial Intelligence Technology. Public Health Nutr., 1–12. 3. Fang, S., Zhu, F., Jiang, C., Zhang, S., Boushey, C.J., and Delp, E.J. (2016, January 25–28). Delp A Comparison of Food Portion Size Estimation Using Geometric Models and Depth Images. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA. 4. Rahman, M.d.H., Li, Q., Pickering, M., Frater, M., Kerr, D., Bouchey, C., and Delp, E. (2012, January 25–29). Food Volume Estimation in a Mobile Phone Based Dietary Assessment System. Proceedings of the 8th International Conference on Signal Image Technology and Internet Based Systems, Sorrento, Italy. SITIS 2012r. 5. Segmentation of Colour Food Images Using a Robust Algorithm;Mery;J. Food Eng.,2005
Cited by
3 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|