A facial recognition demonstration using , , python and the drone , from . Keras Tensorflow Tello DJI here: Source code https://github.com/cleuton/drone-facerec This project is based on two other Github projects: , created by ; FaceREC - CNN implementation of facial recognition me , created by . EasyTello Ezra Fielding Overview I ran this using (Linux). To run using or some fixes must be done. Be my guest to convert it and submit a pull request! Ubuntu MS Windows MacOS This software controls a Tello drone, capturing the video streaming generated by its onboard camera. He intercepts each frame and tries to recognize faces that appear on the image, using a that I trained, based on images from the dataset , and some pictures of mine. Convolutional Neural Network Model Labeled Faces in the Wild It is a proof of concept of the use of artificial intelligence with drones and, why not, Internet of Things (IoT). Note that performance can be low, after all, I am using a laptop and a cheap and simple drone. With more resources, better performance of the solution is possible. To use this solution, you will need: A , with Firmware 1.4 or higher; drone Tello A facial recognition model, generated by the project ; FaceRec_CNN The Dlib face prediction file: (see ); shape_predictor_68_face_landmarks.dat FaceRec_CNN I will teach you how to do these 3 things. Before using There are a few things you need to do before using this application. First it was made exclusively for Drone Tello, but it applies to any other drone that has a python programming API. The second thing to do is to be able to control your drone using only the original project and then try to use this project here. Why? Well, it's not as simple as it looks ... easyTello If you just bought your Tello drone, you need to check which version of it has. This can be done through the app (if you haven't downloaded it, it's better to do it now). Firmware Tello Then, turn on the drone, connect to his WiFi network with your Smartphone (TELLO ...), open the app and click on the settings button, as in the image: Then, click on : More Click on the three dots button: Now, check the firmware version number. If it's 1.3... then you'll need to update it! The process is done in two parts: Download the firmware to the Smartphone and update on the drone. To download the new firmware version, disconnect from the drone's WiFi and connect to the Internet. Using the Tello app, click the button to download the firmware. After downloading, connect to the drone's WiFi and click the button again. Update Update After the process, check the new firmware version. : An indication that the API is out of date is when the drone does not recognize the command ! Warning streamon If you have updated the firmware correctly, I recommend that you try to run the original project, following the instructions in that repository. This confirms that the tracking API is working correctly. Then, you can download and run this repository here. easyTello That done, clone this repository and create an environment using the ( : Anaconda script ./ conda-env.yml) conda env create -f conda-env.yml Activate that environment before running this project: conda activate drone-facerec Creating a model file To do facial recognition, it is necessary to train a neural network. I didn't insert the original project here, not even the model file that I trained. I recommend that you clone the original project and train your own model, and then copy the file created to the folder, within this project. FaceREC_CNN HDF5 easyTello Download and extract the , and place it in the FaceRec_CNN project and also inside the folder for this project! file shape_predictor_68_face_landmarks easytello Generating a model is simple: 1. Get multiple photos of the same person's face Use and download several photos of the same person, copying to the folder the FaceRec_CNN project; Labeled Faces in the Wild raw Name the files in this pattern: john-doe.nnnn.jpg (do not use spaces, number photos of the same subject, separating them with dots). For example: "bill-clinton.0001.jpg"; Take several photos of your face (and whoever else you want to recognize) and save it in the folder following the same nomenclature as in step "1/2"; raw Make sure that there is only one face on each training photo! If there is more than one face, crop the photo; 2. Convert the photos The script will rotate and crop the faces, transforming them into monochrome images. It will separate in training and test photos (folders "train" and "test") according to the variable . If it stays at 30%, then 70% of the images will be for training and the others for testing; trainCNN.py train_test_ratio = 0.3 Write down the categories found! The program will display a vector with the names found. Note to change in the prediction script( ) and in the recognition script for this project ( ); predict.py faceprocessor.py ./ easytello / faceprocessor.py If there are 4 people in your folder, it has to separate 4 people in and 4 people in . With few images, fewer people can be in and this will result in error. raw train test test 3. Get people's names and model file People's names are displayed on the console after training. Copy this vector and change it in ; faceprocessor.py The model file will be named copy it to the folder of this project; 'faces_saved.h5' easytello Before trying with the real drone, I suggest that you run a local test using your webcam: ffmpeg -f video4linux2 -s x480 -r -i /dev/video0 -vcodec libx264 -f  h264 -an udp: 640 15 //localhost:11111 Use the program that I included ( ) to capture the images and do the recognition. If everything works, if he recognizes you, then everything is ready to run on the Drone. facetest_ffmpeg.py ./ facetest_ffmpeg.py If you want, you can run this project with FFMPEG: python teste.py Controlling the drone : To do facial recognition, do not use the original Easytello project! I modified the project to communicate with my facial recognition script ( ). Warning faceprocessor.py ./ easytello / faceprocessor.py The ( ) script controls the drone. It basically does 2 things: puts it in command mode and starts capturing video. But you can do so much more! You can make the drone take off, go forward, or turn around and then land. There are some commented commands that you can use. If you want to know more about the commands that Tello accepts, . teste.py ./ teste.py see the documentation : Do not try to connect your smartphone to the drone, as it only serves a single connection! Warning Make sure you have placed the file in the easyTello folder! Make sure the name is the same as it is in the file ( ) h5 faceprocessor.py ./easytello/faceprocessor.py : If you make the drone take off, make sure there is room for it! If it hits the ceiling or walls, it could be damaged! And be careful when trying to use Tello outdoors (not recommended). If he moves more than 10 meters, he may be out of range of WiFi and collide with something. Theoretically, if he loses contact, he lands automatically. Warning Previously published at https://github.com/cleuton/drone-facerec/blob/master/english_version.md

Read my new book: NeuralJava

Too Long; Didn't Read

Get hands-on learning from ML experts on Coursera 

Facial Recognition On Drone

Too Long; Didn't Read

Company Mentioned

@cleuton-sampaio

RELATED STORIES