Bots and Biometrics

Conversations with a Robot

Let’s imagine a few conversations that people can have with virtual assistants or bots.

For example, a lady named Claire wants to know the weather forecast. — Claire: Ok Robot, what do you do? —Robot: Hello! What’s your name? — Claire: Ok Robot, Claire. —Robot: What can I do for you, Claire. — Claire: Ok Robot, how is the weather in Edinburg today.

Claire comes back a moment later. — Claire: Ok Robot, tell me the weather in Inverness now. —Robot: Hello! What’s your name?

Claire doesn’t understand.

Now Claire wants to know her last bank statements. — Claire: Ok Robot, I want to get my last bank statements —Robot: Hello Claire! It’s been a while. To Help you with this, I need you to add your bank account details.Claire proceeds adding her bank details and she gets her bank statements using a ping number.

Claire forgot something to do with the bank and comes back to the conversation. — Claire: Ok Robot, I need to transfer some money to my husband Jamie. —Robot: Of course Claire, add your 4 digit pin number. — Claire: I have just done it 5 mins ago 😞

Claire’s daughter named Brianna, who is 16 years old, try to access her mother agenda. Brianna wants to schedule a meeting for her mother, on Friday evening, to make a party with her friends at home. — Brianna: Ok Robot, I want to add an event on Friday. —Robot: Sure Claire, which title? — Brianna: Ok Robot, Salary review with Frank (the boss). —Robot: title Added, What time will it be? — Brianna: Ok Robot, at 17.00 for 2 hours. (I will bring all my friends to the party). —Robot: Event Added this Friday from 17.00 to 19.00. Have nice evening Claire.

When Claire finds out the whole story, she wants to … drink.

In previous cases, we can see the difficulties that Bot user face in a regular conversation. At context level, access and identity impersonation.

Why artificial intelligence is not cleverer?Could a bot identify the person without asking the name over and over?

What is a biometric test?

The biometrics are tests based on people physical patterns. These tests allow differentiating one person from other, and establish a profile of that person. For example, calculate the age, sex and mood, among others.

Procedures

Depending on the results of the biometric test, we established the next classification:

• Profiling: Obtain a person metrics like age, sex or mood. Using predefined stored patterns to compare with.
• Verification: To probe if a user is who says she/he is.
• Identification: Given a group of previously categorise people, check if the person belongs to the group.

We can classify biometrics tests attending the type of essay and where is developed:- The fingerprints are one of the most known and old biometrics essays. It is required to have a physical device for the analysis.- A vocal biometric test is needed an audio file.- A facial biometric requires image support.- An image and sound biometric need video support.- For eye-retina, it is required an eye-retina reader. ( Which is believed that produces eye injuries).

What is that for

There is a conversation with people to be handled. The more “legal” information we have from the people “with their consent” the better.If the user is a woman or a man, is called Rose or John, is an adult or an infant, is happy or troubled, is something it can be used to make the conversation more human understandable.

The more “legal” information we have from the people “with their consent” the better.

Extracting people data from the conversation we have more context information and this can be used in the conversation puzzle to provide wiser responses.

Analysing the context type in a conversation, there are:

• User context. If the user is logged into the system, there is important information to check there. Also if the bot runs into an App that extracts system data, for example, the device type, brand, SO, version, among others.
• Conversation context. Linguistic part of the conversation. Current Domain, entities and intents.
• Historic context. Based on the old stored conversation history of the user.
• Inferred context. What surrounds the talk. Day of the week, user sex, user age, user mood, voice type.

We look for giving the best responses, more contextualised and more emphatic. With the biometric tests, the inferred context can be enriched and facilitate the bot to decide a better response.

How biometrics work

For the most popular biometrics as the fingerprint, facial recognition and voice recognition the process will vary depending on the operation.

To get a user profiling we don’t need to train the bot because we don’t need to associate the person with the model but extract the personal characteristics.

Although this implies that verification and identification can be perceived as more invasive than profiling. Those tests steal more attention from the user. Even can be necessary for the user to be in front of the device that is capturing our model for an amount of time.

How to include biometrics in my bot

One idea would be to accommodate the conversation and screens for the person who is talking to the bot. The user could train the models and do the tests following the Bot orientations within the talk.Identify the people with workflow in the conversation is good. Although, the user can reject the idea of being boring or unneeded in the moment of the conversation.

But a spoonful of sugar helps the medicine go down. Ideally reducing the invasion as much as possible. In the case of getting the user profile will be easier because training is not required.

How can we improve the process

We should think about creating a clear strategy for every single case.However, this is not that easy and there are dependencies.

When facing the problem of solving all these variables equation there are few tips we can use:

Asynchronous test

Not all the test need to be within the conversation. We could identify the user with the voice. For example, in a different moment and not disturbing the conversation. When starting a conversation we can catch the user’s voice in any phrase and profile the user without interfering.

Combined tests

Often, the identification and verification can be crucial in our system. Instead of putting all your chances to one card you can combine several biometrics tests at a different time. For example, voice recognition and facial in a different time to authorise a payment in a safer way.

Group biometrics

If your bot has any functionality for groups or can be used for several people at the same time where the security is not very important. You can add a bonus to your conversation. Imagine that your bot recommends restaurants for groups. You can take a picture of the group with a service that gets the gender, age and mood of the group member for a more precise recommendation.

Impersonation (spoofing)

if your bot uses critical systems where the security is something vital. If you are going to implement or to pay a recognition service it is important for the system to detect impersonation. For example, in the case of images. If the identification system implemented in our bot would not have spoofing detection, it would be possible to identify any person with a picture of the authorised user (model) and have access to the critical systems.

Looking at the future

We also made a small research of some futuristics biometrics very interesting and varied. Here are some resutls:

• Veins way. With the finger and palm veins way, can be established patterns because everyone has a unique veins path. https://whatis.techtarget.com/definition/palm-vein-recognition

• Ears shape. Our ear shape can be also used for human detection. The shape of the ears is single. For example, can be used in mobile phones during calls. https://www.newscientist.com/article/dn7672-ear-biometrics-may-beat-face-recognition/

• Body smell. The smell of each of us can be used to identify people. The US army wants to use it to detect potential targets at the distance. https://www.popsci.com/technology/article/2010-04/army-wants-smelling-sensors-can-id-potential-perps-afar
• Behavioural detection. Presence and people behaviour can be detected through electromagnetic waves. https://www.aerial.ai/services/
• Behavioural computer detection. Our way of typing, clicking and moving the mouse can be measured and later used to identify or verify certain users. As it does this project https://www.typingdna.com/.

Conclusion

It looks like the biometrics inclusion in this world is something for the future. Although, we are all using it already in daily routines as the fingerprint with the mobile phone. Now all these ideas are there available to be used in our Bots. We only need to learn when, where and how to include them.

Hope you liked it.Thanks for reading.

Iris biometry by Pablo Alvarez, more details at www.penkorama.com.