Introductory Guide To Real-time Object Detection with Python

Researchers have been studying the possibilities of giving machines the ability to distinguish and identify objects through vision for years now. This particular domain, called Computer Vision or CV, has a wide range of modern-day applications. From being used by autonomous cars for object detection on roads to complex facial and body language recognitions that can identify possible crimes or criminal activities, CV has numerous uses in today’s world. There is no denying the fact that Object Detection is also one of the coolest applications of Computer Vision. If you haven't yet started with Python or you are't familiar with OpenCV then refer this 240+ notes and free Python Cheat Sheet OpenCV Python Tutorial Modern-day CV tools can easily implement object detection on images or even on live stream videos. In this article, we will look at a simple demonstration of a real-time object detector using TensorFlow. Setting Up A Simple Object Detector Pre-requisites: Tensorflow >= 1.15.0 Install the latest version by executing pip install tensorflow We are now good to go! Setting Up The Environment Step 1. Download or clone the TensorFlow Object Detection Code into your local machine from Github Execute the following command in the terminal : git clone https://github.com/tensorflow/models.git If you don’t have git installed on your machine you can choose to download the zip file from . here Step 2. Installing the dependencies The next step is to make sure that we have all the libraries and modules that we need to run the object detector on our machine. Here is a list of libraries that the project depends on. (Most of the dependencies comes with Tensorflow by default) Cython contextlib2 pillow lxml matplotlib In case if you find any of the module missing just execute pip install in your environment to install. Step 3. Installing Protobuf compiler Protobuf or Protocol buffers are Google’s language-neutral, platform-neutral, extensible mechanism for serializing structured data. It helps us define how we want our data to be structured and once structured it lets us easily write and read the structured data to and from a variety of data streams and using a variety of languages. This is also a dependency for this project. You can learn more about Protobufs . For now, we will install Protobuf in our machine. here Head to https://github.com/protocolbuffers/protobuf/releases Choose the appropriate version for your OS and copy the download link. Open your terminal or command prompt, change directory to the cloned repository and execute the following commands in your terminal. cd models protocolbuffers releases v3. /research \ wget -O protobuf.zip https:/ /github.com/ /protobuf/ /download/ 9.1 /protoc-3.9.1-osx-x86_64.zip \ unzip protobuf.zip Make sure that you decompress the protobuf.zip file inside models/research directory Note : Step 4. Compiling the Protobuf compiler Execute the following command from the research/ directory to compile the Protocol Buffer. object_detection/protos/* . ./bin/protoc .proto --python_out= Implement Object Detection in Python Now that we have all the dependencies installed, let’s use Python to implement Object Detection. In the downloaded repository, change directory to . In this directory, you will find an ipython notebook named . This file is a demo for Object detection which on execution will use the specified ‘ ’ model to classify two test images provided in the repository. models/research/object_detection object_detection_tutorial.ipynb ssd_mobilenet_v1_coco_2017_11_17 Given below is one of the test outputs: There are minor changes to be introduced to detect objects from a live stream video. Make a new Jupyter notebook with in the same folder and follow along with the code given below. In [1]: numpy np os six.moves.urllib urllib sys tarfile tensorflow tf zipfile distutils.version StrictVersion collections defaultdict io StringIO matplotlib pyplot plt PIL Image sys.path.append( ) utils ops utils_ops StrictVersion(tf.__version__) < StrictVersion( ): ImportError( ) import as import import as import import import as import from import from import from import from import as from import # This is needed since the notebook is stored in the object_detection folder. ".." from import as if '1.12.0' raise 'Please upgrade your TensorFlow installation to v1.12.*.' In [2]: get_ipython().run_line_magic( , ) # This is needed to display the images. 'matplotlib' 'inline' In [3]: utils label_map_util utils visualization_utils vis_util # Object detection imports # Here are the imports from the object detection module. from import from import as In [4]: MODEL_NAME = MODEL_FILE = MODEL_NAME + DOWNLOAD_BASE = PATH_TO_FROZEN_GRAPH = MODEL_NAME + PATH_TO_LABELS = os.path.join( , ) # Model preparation # Any model exported using the `export_inference_graph.py` tool can be loaded here simply by changing `PATH_TO_FROZEN_GRAPH` to point to a new .pb file. # By default we use an "SSD with Mobilenet" model here. #See https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md #for a list of other models that can be run out-of-the-box with varying speeds and accuracies. # What model to download. 'ssd_mobilenet_v1_coco_2017_11_17' '.tar.gz' 'http://download.tensorflow.org/models/object_detection/' # Path to frozen detection graph. This is the actual model that is used for the object detection. '/frozen_inference_graph.pb' # List of the strings that is used to add correct label for each box. 'data' 'mscoco_label_map.pbtxt' In [5]: opener = urllib.request.URLopener() opener.retrieve(DOWNLOAD_BASE + MODEL_FILE, MODEL_FILE) tar_file = tarfile.open(MODEL_FILE) file tar_file.getmembers(): file_name = os.path.basename(file.name) file_name: tar_file.extract(file, os.getcwd()) #Download Model for in if 'frozen_inference_graph.pb' in In [6]: detection_graph = tf.Graph() detection_graph.as_default(): od_graph_def = tf.GraphDef() tf.gfile.GFile(PATH_TO_FROZEN_GRAPH, ) fid: serialized_graph = fid.read() od_graph_def.ParseFromString(serialized_graph) tf.import_graph_def(od_graph_def, name= ) # Load a (frozen) Tensorflow model into memory. with with 'rb' as '' In [7]: category_index = label_map_util.create_category_index_from_labelmap(PATH_TO_LABELS, use_display_name= ) # Loading label map # Label maps map indices to category names, so that when our convolution network predicts `5`, #we know that this corresponds to `airplane`. Here we use internal utility functions, #but anything that returns a dictionary mapping integers to appropriate string labels would be fine True In [8]: graph.as_default(): tf.Session() sess: ops = tf.get_default_graph().get_operations() all_tensor_names = {output.name op ops output op.outputs} tensor_dict = {} key [ , , , , ]: tensor_name = key + tensor_name all_tensor_names: tensor_dict[key] = tf.get_default_graph().get_tensor_by_name(tensor_name) tensor_dict: detection_boxes = tf.squeeze(tensor_dict[ ], [ ]) detection_masks = tf.squeeze(tensor_dict[ ], [ ]) real_num_detection = tf.cast(tensor_dict[ ][ ], tf.int32) detection_boxes = tf.slice(detection_boxes, [ , ], [real_num_detection, ]) detection_masks = tf.slice(detection_masks, [ , , ], [real_num_detection, , ]) detection_masks_reframed = utils_ops.reframe_box_masks_to_image_masks( detection_masks, detection_boxes, image.shape[ ], image.shape[ ]) detection_masks_reframed = tf.cast( tf.greater(detection_masks_reframed, ), tf.uint8) tensor_dict[ ] = tf.expand_dims( detection_masks_reframed, ) image_tensor = tf.get_default_graph().get_tensor_by_name( ) output_dict = sess.run(tensor_dict, feed_dict={image_tensor: image}) output_dict[ ] = int(output_dict[ ][ ]) output_dict[ ] = output_dict[ ][ ].astype(np.int64) output_dict[ ] = output_dict[ ][ ] output_dict[ ] = output_dict[ ][ ] output_dict: output_dict[ ] = output_dict[ ][ ] output_dict : def run_inference_for_single_image (image, graph) with with as # Get handles to input and output tensors for in for in for in 'num_detections' 'detection_boxes' 'detection_scores' 'detection_classes' 'detection_masks' ':0' if in if 'detection_masks' in # The following processing is only for single image 'detection_boxes' 0 'detection_masks' 0 # Reframe is required to translate mask from box coordinates to image coordinates and fit the image size. 'num_detections' 0 0 0 -1 0 0 0 -1 -1 1 2 0.5 # Follow the convention by adding back the batch dimension 'detection_masks' 0 'image_tensor:0' # Run inference # all outputs are float32 numpy arrays, so convert types as appropriate 'num_detections' 'num_detections' 0 'detection_classes' 'detection_classes' 0 'detection_boxes' 'detection_boxes' 0 'detection_scores' 'detection_scores' 0 if 'detection_masks' in 'detection_masks' 'detection_masks' 0 return In [8]: cv2 cam = cv2.cv2.VideoCapture( ) rolling = (rolling): ret, image_np = cam.read() image_np_expanded = np.expand_dims(image_np, axis= ) output_dict = run_inference_for_single_image(image_np_expanded, detection_graph) vis_util.visualize_boxes_and_labels_on_image_array( image_np, output_dict[ ], output_dict[ ], output_dict[ ], category_index, instance_masks=output_dict.get( ), use_normalized_coordinates= , line_thickness= ) cv2.imshow( , cv2.resize(image_np,( , ))) cv2.waitKey( ) & == ord( ): cv2.destroyAllWindows() cam.release() import 0 True while 0 # Actual detection. # Visualization of the results of a detection. 'detection_boxes' 'detection_classes' 'detection_scores' 'detection_masks' True 8 'image' 1000 800 if 25 0xFF 'q' break EndNote When you run the Jupyter notebook, the system webcam will open up and will detect all classes of objects that the original model has been trained to detect. For more Project Ideas refer Top 25 Computer Vision Project Ideas for 2020 . Happy Learning!!