Using Image Classification for Fitness and Dieting Apps

Washboard abs, buff biceps, or a curvy figure — a body shape that most of us probably desire. However, let's be honest: We're too lazy to get it.

Hitting the gym is a great choice to getting ourselves in shape, but paying attention to what we eat and how much we eat requires not only great persistence, but also knowledge about what goes in food.

The food recognition function can be integrated into fitness apps, letting users use their phone's camera to capture food and displaying on-screen details about the calories, nutrients, and other bits and pieces of the food in question. This helps health fanatics keep track of what they eat on a meal-by-meal basis.

The GIF below shows the food recognition function in action.

Technical Principles

This fitness assistant is made possible thanks to the image classification technology which is a widely-adopted basic branch of the AI field. Traditionally, image classification works by initially pre-processing images, extracting their features, and developing a classifier. The second part of the process entails a huge amount of manual labor, meaning such a process can merely classify images with limited information. Forget about the images having lists of details.

Luckily, in recent years, image classification has developed considerably with the help of deep learning. This method adopts a specific inference framework and the neural network to classify and tag elements in images, to better determine the image themes and scenarios.

Image classification from HMS Core ML Kit is one service that adopts such a method. It works by: detecting the input image in static image mode or camera stream mode → analyzing the image by using the on-device or on-cloud algorithm model → returning the image category (for example, plant, furniture, or mobile phone) and its corresponding confidence.

The figure below illustrates the whole procedure.

Advantages of ML Kit's Image Classification

This service is built upon deep learning. It recognizes image content (such as objects, scenes, behavior, and more) and returns their corresponding tag information. It is able to provide accuracy, speed, and more by utilizing:

• Transfer learning algorithm: The service is equipped with a higher-performance image-tagging model and a better knowledge transfer capability, as well as a regularly refined deep neural network topology, to boost accuracy by 38%.
• Semantic network WordNet: The service optimizes the semantic analysis model and analyzes images semantically. It can automatically deduce the image concepts and tags, and supports up to 23,000 tags.
• Acceleration based on Huawei GPU cloud services: Huawei GPU cloud services increase the cache bandwidth by 2 times and the bit width by 8 times, which are vastly superior to the predecessor. These improvements mean that image classification requires only 100 milliseconds to recognize an image.

Sound tempting, right? Here's something even better if you want to use the image classification service from ML Kit for your fitness app: You can either directly use the classification categories offered by the service, or customize your image classification model. You can then train your model with the images collected for different foods, and import their tag data into your app to build up a huge database of food calorie details. When your user uses the app, the depth of field (DoF) camera on their device (a Huawei phone, for example) measures the distance between the device and food to estimate the size and weight of the food. Your app then matches the estimation with the information in its database, to break down the food's calories.

In addition to fitness management, ML Kit's image classification can also be used in a range of other scenarios, for example, image gallery management, product image classification for an e-commerce app, and more.

All these can be realized with the image classification categories of the mentioned image classification service. I have integrated it into my app, so what are you waiting for?