I made a talking emoji using regular emojis and JavaScript

Today, while I was working, someone sent an interesting little script in a chat group: http://jsbin.com/nijohi/edit?js,output. The original code was written by Martin Kleppe — kudos to him for the short implementation.

The code is very simple, but the effect is very interesting. The emoji looks like it is talking.

Now we are going to see what the code does. The first part of the code [ ..."😮😀😁😐😑😬" ] transforms the string of emojis into an array. In this way, we can select a singular element from that array.

You are probably wondering why we transform a string into an array to select a character from it. This is because an emoji isn’t a singular character, but normally four bytes (an emoji, in reality, looks like this: “\xF0\x9F\x98\x81”).

With this technique, it is possible to make the JavaScript engine preserve the structure of the Unicode character and split the list of emojis in the right way.

As you can see, we have two valid ways to access the right emoji, and the array one is a lot shorter and easier to remember.

The array method works thanks to the iterator implementation of the String object. Instead iterating through every byte like ""[n], it iterates between every code point (a unicode character is a code point).

Now that the string is an array of well-separated emojis, it is possible to select them one by one to get a working emoji. The emoji is selected sequentially using [new Date%6]. This code returns a different sequential emoji every millisecond, the %6 makes sure that we don't go out of bounds.

Finally, the document.body.innerHTML = "<h1>" + sets the emoji as the only content of the page.

But, as you can see, the emoji doesn’t change so fast and doesn’t change every millisecond, but is still sequential.

This is because

setInterval(_=>{ },99)

executes the function every 99ms, and by making 99 milliseconds pass every time, we make the %6 every time a number decreased by one.

When I wrote this article the code wrote by Martin was still the original.After I published it, I notice he changed the code 2 times, to address 2 errors:

The first error is as didymospl pointed out, 99%6 === 3.This means that if the browser correctly executes the timeout at the right time, only 2 emoticons will be displayed, 0 and 3. I will save you the math, if x%y is a multiple of any divisor (excluded 1) of y you will lose some numbers.A quick way to check if the combination of numbers you choose is correct is to check if the greater common divisor of x and y is 1 (GCD(99,6) === 3, GCD(97,6) === 1).So Martin updated the timeout to 97 that isn’t a multiple of 3 or 2.

Even with this correction the code still run oddly on some browsers, this is because setTimeout on the browser isn’t reliable, if the browser is busy doing anything, it will execute the setTimeout in another moment. To make it more uniform across browser, he opts to make it always random, by using ~~(Math.random()*6) instead of new Date()%6.This is another interesting piece of code, Math.random returns a random number between 0 and ~1 (1 is excluded), so if you multiply it by 6 it returns it between 0 and ~6. The only problem is that this number is a float, and we don’t want to access the emoji at position 1.2, but instead the one at position 1. To make it integer he used the esoteric operator ~, this operator is a bitwise not, in math terms it returns (Math.floor(n) + 1) * -1, normally this operator is used for another trick, check if a number isn’t -1 (very useful with indexOf). But in code golf it has another use, converting anything to an integer, by using it 2 times, it keeps only the Math.floor effect.

If you want to save 11 bytes on your code golf competition, you can replace Math.floor with ~

The previous code was going into reverse. I needed to avoid that, so I need to have a number that increases. To obtain that, I use Math.floor (new Date/delay). Instead of returning the current milliseconds, it returns the current tenth of a second.

Now that the number is progressive, I just need to limit it to a range. For the emojis, I need to stick to the number of emojis (6), and for the text, I need the length of text + 1 (I use +1 to show the latest character, remember length%length == 0).

The result is kind of nice: the emoji moves its mouth and a text appears below. But the movement of the mouth is totally unrelated. I’ve been watching dubbed movies since I was born, and the ones badly dubbed ones have always irritated me, so I have to do something to make that emoji animate in a better way.

The first thing I searched was an image that illustrates the various movements of the mouth for each letter. After Googling for five minutes, I finally find the right image:

Time to map the various emojis to each letter. For that, I need a nice page with all of the emojis that I can possibly need: https://emojipedia.org/apple/.

After selecting one for each unique mouth movement in the picture, I proceed to create the map:

A default emoji for non letters characters ( “ “, “!”, “,” … ) — now I just need to change the code to retrieve the right emoji.

This new code is subdivided in two parts:

1. Finding the character the emoji is currently pronouncing:

The code is simple, because we are using the same code logic that shows the message but selecting only one character. The main difference is the message.toLowerCase(), because I need it to be case insensitive when I'm checking if the character matches with the ones in my emoji map.

2. Selecting the right emoji:

This code first transforms the emoji map into an emoji array. In this way, I can use the emojis as keys and check each value one by one. For example:

The find function will check the values one by one until the function I execute for that value returns true. In our case, the function I’m using is emoji => emojiMap[emoji].includes(character). That simply checks if the set of characters for that emoji includes the character I'm searching for.

If the find doesn’t find anything, it will return undefined, that is a false value. By using || defaultEmoji, I can make my code returns defaultEmoji when find doesn't find anything.

The functionality now works, the emoji speaks correctly, but on the phone it looks terrible (on Android). I want it to work even on mobile, so I need to make the code return an emoji that is equal on every platform. To do that, I’m going to use twemoji.

Twemoji is a library from Twitter to use their emojis everywhere. The library is simple to use, has a method named parse that parses a string of text, and returns a string of HTML where every emoji is an image.

That is perfect for now, and very simple to implement. I first include the script in the page, and I make a small change to convert my emojis into images:

The image is a little bit low quality. Maybe we can do something. Looking at the documentation, I notice I can use SVGs.

Let’s use them:

The emoji now looks fine on the mobile phone, but there’s too much text. To solve that, I split the text I’m showing into words, and I show only the last two:

The trick is simple. To split the words, I use .split(' '), which breaks a string into an array of strings separated by a space.

const words = message.substr(0, Math.floor(new Date / delay)%(message.length+1)).split(' ')

Then, I obtain the current word the emoji is saying using .pop()

const current = words.pop()

I add everything to the page, popping out the word before the latest one.

document.body.innerHTML = "<h1>" + ... + "</h1><h3>" + words.pop() + "</h3><h2>" + current

Finally, the only minor thing is left is that on the image with the mouth position, there are not only singular characters, but also combinations, like sh and th.

It would be amazing if the script could catch them.

To catch those two character combinations, I need to look at not only the current character, but also the previous one and the next one.

The code I just wrote creates three variables: the current character, the current character together with the previous one, and the current character together with the next one.

Now that we have all of these three variables, we need to give priority to a couple of characters. To do that, we first search for the previousDouble on the emoji map. If we find nothing, we search for the nextDouble and finally for the current character.

Finally, a little bit of cosmetic improvement and the talking emoji is done!

There’s still a lot of room for improvement, here a couple of ideas if you want to extend this script further:

1. Use the Web Speech API to make the emoji talk for real. This will have several challanges, for example sync the “lips movement” with the voice.
2. Use D3 to animate the transition from one emoji to the another making it more natural and realistic.
3. Stop using emoji and use more expressive images, also recognise more diphthongs. This is easier because the code needs nearly 0 changes.

This story was originally published on codementor.