Moodify in action March was a pretty productive month, all thanks to this major event hosted by Association for Computing Machinery, NIT Surat Student Chapter. Basically build what you can in a month and show it off in a presentation. The key was innovation and implementation in all aspects including programming involved, design and even the code itself. Well, the idea (which Dhanush came up with) was an app that recognizes your mood from a snap taken and generates a suitable playlist in the music player. Pretty neat, eh? Implementation To start off with, we were actually lost for quite a while cause this idea used technologies that were less documented and of course we had no tutorials to follow. But with research and experimentation, we decided to use with Python to create scripts that could extract mood from a captured image, and since coding was already started with Python, was chosen as the framework to use for wrapping all this as a WebApp. OpenCV Flask The WebApp The development work was split into multiple components, like a Camera, Music Player and the server which handles the routes and database. So the front end was programmed with to give it a app like flow while the UI was designed over the CSS library. AngularJS Materialize Talking about individual components, The Camera and the Music Player : WebcamJS was mainly used in handling the cam with just the frontend, with generated base64 image passed into the server. Camera : SoundManager 2 API implemented with AngularJS used to handle audio files Music Player : It’s a basic Flask server which connects to the MongoDB database and also handles routes. The basic routes for this app was: Server : The main or initial route which redirects to the Cam-app /index : Links to the camera, saves snap to server /cam : Links to the music player, also takes in mood from URL as a parameter to get only mood specific songs from DB. /player : Runs a python script which uses snap in server to extract the mood and return its value /emotion : GET request to MongoDB to return JSON data of the songs /songs WebApp file structure and Database JSON data The App Flow You start at the / , where you can take snaps, retake and proceed. This basically saves the snap to the server and goes to the route to call the Python script that extracts mood and returns it, redirects to / with mood as a parameter. / calls with the mood parameter to return songs as per generated mood. This is displayed in the music player. cam /emotion player player /songs The Mood Recognition Okay, this part isn’t exactly mine, but I’ll try explain it in simple terms the way I understood. Basically, / route gets the image from the cam and stores it in server, on which the mood recognition script runs on. emotion The script initially optimizes the image so that detection happens most accurately, basically cropping it to just the face and then converting to grayscale. Now that the image is ready to work on, OpenCV analyses the contours and detects the features or landmarks of the face, as depicted in the picture. Detecting facial landmarks Using this data, along with machine learning, mood can be identified. Models are made by training on many faces. Multiple faces showing the same mood is fed as training data and so it can identify the nearest mood when it gets a new face. Training data for anger mood This is achieved using OpenCV, more precisely, Haar Cascades for face recognition and dlib Predictors for facial landmark analysis. The Project and Developers The project’s official repository : _amoc-project - Moodify: Recognizes emotion from face, generates a suitable playlist in the music player_github.com ajayns/amoc-project Developers : _ajayns has 14 repositories available. Follow their code on GitHub._github.com ajayns (Ajay NS) _dhanushkamath has 2 repositories available. Follow their code on GitHub._github.com dhanushkamath (Dhanush Kamath) Also, feel free to contact me at ajay.ns@outlook.com, check out my . Do recommend and share my post if you found it useful. GitHub

Flow

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

ACM Month Of Code 2k17: Building Moodify

Too Long; Didn't Read

Coin Mentioned

Ajay NS

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

ACM Month Of Code 2k17: Building Moodify

Too Long; Didn't Read

Coin Mentioned

Ajay NS

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES