So Don’t Panic. Or do? Idk One night, founder and CEO Shon Burton had a crazy dream that he was Iron Man. He was wearing the mechanized super-suit complete with all the rockets, laser beams, and targeting helmet. HiringSolved Anti-climatically, he wasn’t anywhere near a battlefield. He was at an HR conference. Like many of us, Shon had fallen victim to dreaming about work. Disappointed at first, Shon then noticed his helmet was doing something funny. Anywhere he looked, a targeting reticle would lock onto someone’s face and populate a floating field containing all of that person’s professional details. Name, contact information, title, achievements… you name it. When he woke up he thought “this is totally doable”. “We were willing and excited to see what could be done — we have great data and interesting capabilities so we threw it together with the engineering team to see what happens. It became one of our summer labs projects.” Burton said. Meanwhile… A recent about the FBI’s use of facial recognition suggests 1984 has arrived and we are becoming members of The Party. Georgetown Law study Articles with titles like from help paint the picture that Big Brother is here. Cops Have a Database of 117M Faces. You’re Probably in It Wired Does advanced image processing create a “ ” for local law enforcement, FBI, CIA and other federal agencies as Georgetown suggests? perpetual lineup Or did the Tony Starks of the world get there first? “Face recognition error rates have decreased over the last twenty years by three orders of magnitude” — Facebook Research With Big Data Comes Great Responsibility Background on HiringSolved plays primarily in the talent acquisition world. Their engineering team builds products like — a conversational AI that helps recruiters discover candidates on the web. — a Chrome extension that predicts and uncovers information about anyone whose page you visit, and — a Google-esque search engine that crawls the web as well as contracted company candidate databases to assemble and organize everyone into consistent professional profiles. HiringSolved RAI PROPHET TalentFeed Tony Stark vs. The Law Tenets of Facial Recognition “The conventional pipeline consists of four stages: → .” — detect align→ represent→ classify Research at Facebook Detecting: Finding a face The FBI In law enforcement, this stage relies on things like . This creates issues differentiating black people among other things. eye position or skin texture Tech Startup Unlike FBI methods to find the face in an image, face detection step doesn’t use anything except the direction of light. HiringSolved’s This method is known as Histogram of Oriented Gradients or HOG. For the first step in this method, the system desaturates an image making it black-and-white rendering color irrelevant. Then the system replaces all the pixels with arrows based on the procession of light-to-dark where the pixels once were, enabling the system to detect distinct features of a face based on the patterns created. Aligning: Centering with face forward The FBI The FBI has trouble with faces that are obscured or not aligned properly. For an accurate read,FBI systems have to capture a face straight-on. They cannot handle images the center axis. more than 15 degrees off FBI Tech Designed for Mugshots Tech Startup The tech used by can align any face in any position using a two-part aligning method. HiringSolved The first part is an algorithm based on a method of “face-landmark estimation.” This breaks a face down into 68 points which each reference a facial landmark. The second part is an algorithm that conducts “affine transformation.” This rotates, scales, and shears the image so everything is front and center. Any processed image should now have the eyes and mouth in roughly the same spots so it is prepped for encoding. Representing: Converting image into numeric set The FBI The FBI may not do this crucial time-saving step. A from their face recognition suggests they may skip representation altogether: document program “All search submissions to the IPSFRP system MUST include a single frontal photo of the search subject. Further, face images must be compressed within the search transaction using JPEG encoding.” Also, their system compares pre-determined measurements of features like distance . between the eyes Tech Startup Matching millions of image files to each other takes forever so tech startups use a set of numbers to represent and processes matches much much faster. system uses a machine technique that decides which 128 measurements on a face to log, unlike FBI systems that use measurements determined by humans. HiringSolved’s learning then takes the aligned face and captures facial measurements for representation. This capture is known as “embedding” in machine learning circles. HiringSolved For the best results in the following classification step, the rule-of-thumb is to embed 20+ images per person for higher accuracy. Classifying: finding the closest match in database The FBI The FBI doesn’t find just one match. Instead the system finds the top 50. Out of those top 50 matches, it claims that one of them is a match only 85 percent of the time, according to an Electronic Frontier Foundation ( ) study. EFF The EFF report comments that a system like this “will return a lot of images of the wrong people.” The FBI relies on databases consisting solely of . So, their system can only really reference one or two photos of a person which lowers their measurement accuracy. driver’s license photos and mugshots Tech Startup has a similar issue to the FBI since their image database also has a low per-person count. HiringSolved Instead of relying on a sub-par image volume, engineers elected to build a live trainer for the classifier. The trainer boots on a MacBook, asks for the user’s name and company, and then captures a short video. This takes 50 images of the user for the embedding. Once the classifier is trained using the 50 images, it can be accessed by a computer’s camera via a server. Since is working towards a mobile facial recognition product, they built the classifier app for a Droid Turbo 2. HiringSolved || v After training the classifier with multiple people’s faces, the device is able to find a match in two seconds! Fear the FBI? Should we fear our government overlords? No. Not yet. While the FBI has access to face data, it’s too thinly-spread and they aren’t able to use it efficiently. If a small tech startup can can develop highly accurate facial recognition software for use on a camera phone relatively cheaply, rivaling the that Facebook currently touts, we need to trust the Tony Starks (and Facebooks, Snapchats, or Googles) of the world are using their powers for good. 98% accuracy rate Some entities, like Facebook, have a major inherent advantage with a high volume of indexed images while others struggle to create gimmicks that collect face data (see Microsoft’s “ ” app). How Old Do I Look? Getting people to give you a facial embedding is hard. If only there was . We could make it fun. Free. And social. Let people invite their friends. Spread it. Let them put silly cartoon items on their face. Then promise we’re deleting it all. a way to coerce people To be fair, Snapchat they can’t “Recognize a Specific Face”. claims So why worry? It’s not like they have a engineering wing or something like that. secret ( ) Disclaimer: I wrote this story in the fall of 2016 so there’s a good chance that federal agencies have begun co-opting the latest face rec. techniques by now.
