The ability to gather information from images has profound business potential. And, well, it can also just be fun. In this article, I’ll outline how we used to rapidly build an image object detection feature for a social media startup. Amazon Rekognition What Is Image Object Detection? Simply put, image object detection is the process of detecting and extracting information about entities in a given image. This involves detecting objects, activities, places, animals, products, etc. Image object detection has a wide array of use cases across a variety of industries. Major sectors such as banks, insurance, social media, dating apps, news agencies, and FinTech use object detection in some form or another. Our Use Case Recently, we were tasked with building an image object detection feature for a social media startup. The use case was simple — users should be able to select some of their favorite photos and submit them to be featured on one of the startup’s social media pages. The social media marketing team needed a way to search through the image submissions for certain themes — such as photos of the ocean, popular landmarks, animals, music concerts, etc. Image Analysis Using Deep Learning Analyzing images and classifying them based on scenery and objects within the image is no simple task. Human sight is nothing short of remarkable, and building an application that’s able to replicate the brain’s ability to detect objects is immensely complex. There is an entire computer vision industry devoted to doing just that. Performing object detection from scratch is typically a multi-step process that involves: Collecting images and labeling the objects within Training the ML models Using the trained models to perform the analysis Performance tuning And, repeat… Our aim for this feature, like all others on the project, was to build it quickly and test its efficacy in a production environment as soon as possible. Furthermore, we didn’t want to devote development resources to building a solution from the ground up when we could leverage existing cloud services. Cue Serverless — the startup’s entire backend is fully Serverless and event-driven. With this architecture, we’re able to have teams of developers that only need to focus on features that the social media app from others. Serverless also enables us to build highly scalable services whilst also only paying for exactly what we use — an important consideration for a scaling startup. differentiate So to achieve this feature, we used Amazon Rekognition — a fully Serverless image and video analysis service. Using Rekognition, we were able to develop this complex and critical workflow in a matter of . Let’s dive into it. hours What Is Amazon Rekognition? Amazon Rekognition is an AWS Serverless offering that uses deep learning to perform image and video analysis. Being fully Serverless means that with Rekognition we don’t need to worry about the complexity of the underlying infrastructure; we pay only for what we use it provides us with pre-written software for image and video analysis tasks. Rekognition offers a range of features, including image label detection, face detection, celebrity detection, content moderation, and text detection. and The best part? Rekognition abstracts away the heavy lifting of building, training, and analyzing deep learning models. Image and video analysis is quick and simple, with minimal set-up necessary. We didn’t need to worry about building and training our own datasets and provisioning server capacity so that our service would scale. All we needed to worry about was integrating. Architecture The architecture is straightforward. Our mobile app uploads images from users’ phones into an S3 bucket. The upload to S3 then triggers a Lambda function which in turn calls the Rekognition API and stores the results in DynamoDB for querying. Serverless image object detection architecture diagram using AWS The Code Writing code is fun, right? Well, writing less code is even more fun. Rekognition exposes a set of APIs that you send image data to which perform the analysis and return the results. For our use case, we used the API. detect-labels A simplified version of our Serverless framework Infrastructure as Code file looks like this: //serverless.yamlfunctions:
  imageLabelDetection:
    handler: image-label-detection.handler
      events:
        - s3:
          bucket: my-image-bucket
          event: s3:ObjectCreated:*
          existing: true
      iamRoleStatements:
        - Effect: Allow
          Action: rekognition:DetectLabels
          Resource: "*"
        - Effect: Allow
          Action: s3:GetObject
          Resource: arn:aws:s3:::my-image-bucket Our Lambda code simply calls the Rekognition API and stores the results in DynamoDB, but you can use whatever makes sense for your use case. We obtain the S3 bucket name and the image’s object name from the S3 event and pass those into the function of the . detectLabels Rekognition SDK We also pass in two optional parameters (MaxLabels and MinConfidence) to specify the confidence level threshold and a maximum number of labels that we want returned. In the example below, we will only get up to 20 labels in the response, and all labels will have a confidence level of more than 80%. //image-label-detecion.jsconst AWS = require("aws-sdk");const rekognition = new AWS.Rekognition();exports.handler = async (event) => {
  const imageDetails = event.Records[0].s3;  const bucketName = imageDetails.bucket.name;
  const objectKey = imageDetails.object.key;  const rekognitionResp = await rekognition
    .detectLabels({
      Image: {
        S3Object: {
          Bucket: bucketName,
          Name: objectKey,
        },
      },
      MaxLabels: 20,
      MinConfidence: 80,
    })
    .promise();  // Send to data store, e.g. DynamoDB
  // ...
}; Who doesn’t love a picture of a dog? Below is a response for an image that we uploaded to our S3 bucket. As you can see, Rekognition correctly determines that it’s an image of a dog on an outdoor, gravel path (and tells us where in the image the dog is!). Rekognition response (left), uploaded image (right). Final Thoughts So, what are my thoughts after using Rekognition in production for a few months? Here is a list of key takeaways: Fully Serverless — Avoid the undifferentiated heavy lifting of managing underlying infrastructure complexity and use pre-built software and pre-trained models. Performance — Rekognition is quick, around 600ms in most cases. Easy to use — The API is simple and easy to integrate with. Continuous improvement — AWS is constantly maintaining and iterating on Rekognition meaning it will become increasingly better over time. Accuracy — Rekognition takes advantage of Amazon’s broad set of customers; resulting in highly accurate image label detection. Has a generous free tier, so you can trial before deciding. Cost — We pay only for what we use. Rekognition allows us to scale up and down based on business needs. Summary TLDR: Rekognition enabled us to rapidly build an image object detection feature that’s accurate, fast, and scalable.

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

Using Amazon Rekognition to Build an Image Object Detection Feature for a Social Media Startup

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps