Calorie Aware Automatic Meal Kit Generation from an Image

by   Ahmad Babaeian Jelodar, et al.

Calorie and nutrition research has attained increased interest in recent years. But, due to the complexity of the problem, literature in this area focuses on a limited subset of ingredients or dish types and simple convolutional neural networks or traditional machine learning. Simultaneously, estimation of ingredient portions can help improve calorie estimation and meal re-production from a given image. In this paper, given a single cooking image, a pipeline for calorie estimation and meal re-production for different servings of the meal is proposed. The pipeline contains two stages. In the first stage, a set of ingredients associated with the meal in the given image are predicted. In the second stage, given image features and ingredients, portions of the ingredients and finally the total meal calorie are simultaneously estimated using a deep transformer-based model. Portion estimation introduced in the model helps improve calorie estimation and is also beneficial for meal re-production in different serving sizes. To demonstrate the benefits of the pipeline, the model can be used for meal kits generation. To evaluate the pipeline, the large scale dataset Recipe1M is used. Prior to experiments, the Recipe1M dataset is parsed and explicitly annotated with portions of ingredients. Experiments show that using ingredients and their portions significantly improves calorie estimation. Also, a visual interface is created in which a user can interact with the pipeline to reach accurate calorie estimations and generate a meal kit for cooking purposes.


page 1

page 3

page 5

page 7


Automatic estimation of heading date of paddy rice using deep learning

Accurate estimation of heading date of paddy rice greatly helps the bree...

Automatic Grading of Knee Osteoarthritis on the Kellgren-Lawrence Scale from Radiographs Using Convolutional Neural Networks

The severity of knee osteoarthritis is graded using the 5-point Kellgren...

CameraNet: A Two-Stage Framework for Effective Camera ISP Learning

Traditional image signal processing (ISP) pipeline consists of a set of ...

Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Model

This paper studies the fundamental problem of learning energy-based mode...

PupilNet: Convolutional Neural Networks for Robust Pupil Detection

Real-time, accurate, and robust pupil detection is an essential prerequi...

Designing Machine Learning Pipeline Toolkit for AutoML Surrogate Modeling Optimization

The pipeline optimization problem in machine learning requires simultane...

Generating Datasets of 3D Garments with Sewing Patterns

Garments are ubiquitous in both real and many of the virtual worlds. The...

Please sign up or login with your details

Forgot password? Click here to reset