Abdul Kareem, Razia Sulthana ORCID: 0000-0001-5331-1310 , Tilford, Timothy ORCID: 0000-0001-8307-6403 and Stoyanov, Stoyan ORCID: 0000-0001-6091-1226 (2024) Fine-grained food image classification and recipe extraction using a customised Deep Neural Network and NLP. Computers in Biology and Medicine. ISSN 0010-4825 (Print), 1879-0534 (Online) (doi:https://doi.org/10.1016/j.compbiomed.2024.108528)

Preview

PDF (AAM) 46890_ABDUL KAREEM_Fine-Grained_food_image_classification_and_recipe_extraction_using_a_customised_Deep_Neural_Network_and_NLP.pdf - Accepted Version Available under License Creative Commons Attribution. Download (1MB) | Preview

Abstract

Global eating habits cause health issues leading people to mindful eating. This has directed attention to applying deep learning to food-related data. The proposed work develops a new framework integrating neural network and natural language processing for classification of food images and automated recipe extraction. It address the challenges of intra-class variability and inter-class similarity in food images that have received shallow attention in the literature. Firstly, a customised lightweight deep convolution neural network model, MResNet-50 for classifying food images is proposed. Secondly, automated ingredient processing and recipe extraction is done using natural language processing algorithms: Word2Vec and Transformers in conjunction. Thirdly, a representational semi-structured domain ontology is built to store the relationship between cuisine, food item, and ingredients. The accuracy of the proposed framework on the Food-101 and UECFOOD256 datasets is increased by 2.4% and 7.5%, respectively, outperforming existing models in literature such as DeepFood, CNN-Food, Wiser, and other pre-trained neural networks.

Item Type:	Article
Uncontrolled Keywords:	image classification; ingredient identification; recipe extraction; Deep Neural Networks; domain ontology; natural language Processing
Subjects:	Q Science > Q Science (General) Q Science > QA Mathematics Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Faculty / School / Research Centre / Research Group:	Faculty of Engineering & Science Faculty of Engineering & Science > School of Computing & Mathematical Sciences (CMS)
Last Modified:	02 May 2024 14:10
URI:	http://gala.gre.ac.uk/id/eprint/46890

Actions (login required)

View Item

Downloads