Machine Learning for Food Recognition with Android Demo

It seems that Food Recognition has had a lot of attention lately due to HBO’s Silicon Valley episode from season 4, “Intellectual Property”. I’ve heard about it and had to watch it since my current project at work is, you guessed it, Food Recognition. The only difference is that when we started building machine learning powered food recognizer what we had in mind was food variety and a healthy diet. Not that there is something wrong with having a hotdog once in a while.

There was a very nice article explaining the machine learning approach to food recognition How HBO’s Silicon Valley built “Not Hotdog” with mobile TensorFlow, Keras & React Native if you’re interested in that.

What we’ll try to do is an Android App that recognizes way way more than just hot dogs from images. It shouldn’t take more than 15 minutes or so

Azumio Inc. introduced a new API called Calorie Mama AI Food Recogniton. API has a full potential of recognizing food on your plate by using machine learning in the background. In addition to recognizing food it can also recognize packaged goods such as candy bars or canned ham for example.

Why food recognition?

It’s not to be expected that food recognition from photos will make you healthier or skinnier by default. But it sure beats manually inserting nutritional value of your food in an app. So I look at this functionality as hyper helpful assistant that does that for me or at least helps me simplify that process.

Why tracking food is good?

In short: It makes you mindful of what you eat which has many health benefits (Source)

Android

This is my humble attempt at Android App that uses that API. By humble I mean I haven’t developed an Android App in about 6 years or so. A lot has changed since then. Luckily for the better. Android Studio does a nice job guiding you through the first steps. Not having to deal with hardware and software fragmentation (more than 100 different devices with different screen sizes, keyboards, camera buttons etc.) and no software/hardware standardization across devices (carriers can modify the OS itself) I plow on with a smile on my face.

The Request

It’s pretty straight forward. All we need is to do is resize the image to 544x544px and then POST that image in JPEG format:

curl -H -i -F [email protected] https://api-2445582032290.production.gw.apicast.io/v1/foodrecognition?user_key=[YOUR KEY HERE]

The Response

{    "model": "20170209",    "results": [{            "packagedgoods": true,            "group": "Packaged Good",            "items": [{                "nutrition": {                    "totalCarbs": 0.875,                    "protein": 0.0125,                    "calories": 3500,                    "sodium": 0.003,                    "iron": 9.0E-6,                    "sugars": 0.625...

All the nutritional values are in standard International System of Units which in this case means values are in Kilograms.

How to convert vitamins to percentage of Daily Recommended Values?

For instance if we take Vitamin A all we need to know what is the recommended daily intake. We can find the necessary conversion on USDA website.

For vitaminA DRV is 5000 IU. With the help of the USDA calculator we convert that to mcg (5000 IU * 0.3 = 1500 mcg). Note that we used conversion of vitamin A as retinol.

weight (mcg) / 1500 (mcg) = vitamin A %

GitHub Project

igorrendulic/food_recognition_with_calorie_mama_food_recognition_with_calorie_mama - Food Recognition Using Calorie Mama AI API_github.com

I’m a backend software engineer at Azumio Inc. and was/am part of the food recognition project team.