Nutrient and Food Group Prediction as Orchestrated by an Automated Image Recognition System in a Smartphone App (CALO mama): Validation Study

Abstract

Background A smartphone image recognition app is expected to be a novel tool for measuring nutrients and food intake, but its performance has not been well evaluated. Objective We assessed the accuracy of the performance of an image recognition app called CALO mama in terms of the nutrient and food group contents automatically estimated by the app. Methods We prepared 120 meal samples for which the nutrients and food groups were calculated. Next, we predicted the nutrients and food groups included in the meals from their photographs by using (1) automated image recognition only and (2) manual modification after automatic identification. Results Predictions generated using only image recognition were similar to the actual data on the weight of meals and were accurate for 11 out of 30 nutrients and 4 out of 15 food groups. The app underestimated energy, 19 nutrients, and 9 food groups, while it overestimated dairy products and confectioneries. After manual modification, the predictions were similar for energy, accurately capturing the nutrients for 29 out of 30 of meals and the food groups for 10 out of 15 meals. The app underestimated pulses, fruits, and meats, while it overestimated weight, vitamin C, vegetables, and confectioneries. Conclusions The results of this study suggest that manual modification after prediction using image recognition improves the performance of the app in assessing the nutrients and food groups of meals. Our findings suggest that image recognition has the potential to achieve a description of the dietary intakes of populations by using “precision nutrition” (a comprehensive and dynamic approach to developing tailored nutritional recommendations) for individuals.