Selfie Biometrics: Exploring Face Recognition & Liveness Technologies for Mobile Apps

10-12 years ago, contactless payment by smartphone (NFC) seemed to be a fantasy. It's now slowly becoming a reality.

Now you don't even need a smartphone to make a purchase in some offline stores. Now you just need to look at the camera at the checkout. What I'm talking about is selfie biometrics — biometric facial recognition made during transactions (Face Pay). This technology is also used to access a device, a service, an app.

In this article, I explain how Face ID and Liveness face recognition applications work. I think it's time for you to know this because a password-free future awaits all of us. Selfie biometrics will very soon become our verification standard, displacing even fingerprints and iris scans, as face scans don't require any direct contact with a user. 

Face ID is a biometric facial recognition technology for face identification in order to confirm a person's identity. The technology applies methods of 3D and 2D scanning of a user's face, analyzing the details of the face image, and matching them to a reference from the database.

Simply put, each person has a unique facial structure. Special software can analyze it by matching it with information in the database to later identify who you are. Comparison of facial patterns occurs using programs based on artificial convolutional neural networks.

Facial recognition technology (FRT) has been actively used in law enforcement as a tool to track down criminals. Banks and offices use FRT in Physical Access Control System (PACS).

Besides, FRT finds use in surveillance systems of airports and other public places, as well as in medicine and genetics to detect genetic disorders (e.g., Face2Gene and DeepGestalt frameworks).

In mobile phones, the technology is used to unlock a device, access apps, and make monetary transactions (payment, transfer). As for Android smartphones and tablets, facial recognition has been used for about a dozen years in these devices. However, the method used there for identification is not safe because it uses a 2D recognition algorithm. That is, only a photo, not a 3D image, is enough to unlock a phone, so this algorithm is easy to deceive.

Front camera quality is a determining factor here, the same as the complexity of the algorithm used to extract details of a face. More expensive smartphones use convolutional networks, unlike low-budget options (like 360 Face Unlock from Huawei and OnePlus quick unlock technology).

Three-dimensional facial recognition in mobile was pioneered by Apple Inc., unveiling the Face ID feature in 2017 as part of the iPhone X launch. It is Apple that has come up with the name that has now become a household name for facial identification technology.

Various companies now use the term “Face ID” in  their service solutions for banking, retail, delivery and logistics, access control systems (e.g. Face ID Biometrics, Evergreen). All software giants have their facial recognition solutions today: Google (FaceNet); Facebook (DeepFace); Microsoft, IBM and Megvii (FACE++); Amazon (Rekognition); Academia (GaussianFace).

Still, Apple Face ID remains the most popular real-world case. To biometrically scan the face shape and recognize it when unlocking, the iPhone is powered by the TrueDepth camera, equipped with two infrared projectors that highlight the face points and save them to memory as mathematical code.

A camera works together with proximity and illumination sensors, a Flood Illuminator backlight system (which can't be seen in the dark), and the Dot Projector. All this creates a highly accurate 3D model of a face and saves the template directly to the processor. A specially locked part of the smartphone's CPU is responsible for saving the Face ID key.

According to the developers, the system is capable of self-learning and adapting to a user, without paying attention to clothing, eyeglasses, and age-related changes in a face.

Face ID can be used to authorize purchases using the Apple Pay payment system. Apple claims that Face ID verifies compliance using facial structure data that can't be read from a printed or digital 2D photo. Sophisticated neural networks protect against scams using masks or other techniques.

Face ID technology can even determine what your eyes are focused on. It only recognizes your face if your eyes are open, and you are looking at the device. This complicates the task of unlocking the device without you knowing (for example, when you're asleep). The chance of another person being able to unlock your iPhone or iPad Pro using Face ID is 1 in 1,000,000.

But how does the system determine whether the person in front of it is real and alive? There is another interesting technology for this — Liveness. It is in charge of determining whether an object is alive. The technology ensures that a user is physically present to be authenticated.

Liveness Detection prevents bots and attackers from using stolen photos, embedded deepfake videos, realistic masks, or other fakes to create or access online accounts. Liveness ensures that only real people can create and access accounts. The step where a user confirms their actual presence has the following purpose.

It helps thwart a spoofing attacker to use a photo, video, or other data to the substitution of the authorized biometric characteristics to gain someone's benefits or access rights. Liveness Detection introduced in the system must comply with the ISO 30107-3 attack detection standard.

Various developers provide society with their Liveness-solutions (FaceTec, ID R&D, VisionLabs), but their working principle is identical. 

Before a user provides an identity document when registering, they must prove that they are real. This is to prevent the attacker from being able to identify by a fake photo or a realistic face mask. Two types of data are required for each user authentication: data about the face (for comparison) and data confirming validity (to confirm that data came from an actual person).

The validity data should be time stamped and only valid for a few minutes and then deleted. Only data about individuals should be kept permanently. New validity data should be collected at each authentication attempt. 

To sum up, the biometric system must determine whether the data came from a living person or from a non-living artifact (an inanimate object that tries to replicate human biometric traits).

3D Liveness analyzes your quick video selfie for the attributes of a living person: these are specific movements of the head and facial muscles in response to commands and their corresponding shadows and glare on the skin texture; micro-changes in facial expressions in motion and static; reaction speed. Once it is proven that a new account belongs to a real person, their biometric data will be saved as a reliable benchmark of their digital identity for later authorization. 

Developer FaceTec — the market leader — speaks of Liveness as a “brick wall for bad actors.” The company suggests using your three-dimensional face to unlock everything from a car door to a bank account. It takes 2 seconds to determine whether it's an alive person or not.

Real users get easy access, while attackers and bots get rebuff with an anti-spoofing AI certified by the NIST/NVLAP lab. This AI is constantly tested using a $100,000 Spoof Bounty (Spoof Reward Program — incentive public safety test designed to secure biometric authentication in the real world, not just in a lab or classroom). 

KZen Networks, a Tel Aviv-based developer of the ZenGo cryptocurrency wallet, wanted to create a “perfect” cryptocurrency management solution. Here is the key requirement. Users’ authentication must verify that they are legit, alive, and physically present at the time of the login request. As Uriel Ohayon (CEO of ZenGo) says, they use FaceTec authentication because it has been “battle-tested on millions of users, including banks”. One “can' t play” with it.

According to Bloomberg, the global facial recognition market is expected to grow by $3.35 billion in 2020-2024, with 29% of the growth coming from the Asia-Pacific region. 

Financial institutions around the world are increasingly encouraging the use of mobile and personal devices for online financial transactions. With the increasing use of consumer electronics for financial transactions, the need for a high level of security also increases as critical user information can be easily hacked. Since the number of smartphone users continues to grow, the demand for facial recognition technology will grow.