Subtitles for Living: AR's Role in Language Translation

The value of language translation is not a hard sell. It is a phenomenon that continues to open up the world in many ways, transcending divisive lines of geography, culture, and race. 

Many believe that the first known instance of translation occurred in the Mesopotamia era when a Sumerian poem called Gigamesh was translated to an Asian language. Since then, human intellect has come a long way, and so has our technology. And as a result, language translation has become a lot less tedious over time, with augmented reality being one of the reasons why. 

I, for one, have always been fascinated by the idea that so many people can communicate the experience of ‘living’, with a unique vocabulary outside my comprehension. 

Even though augmented reality can’t help us learn new languages on a whim, the folks at Google have proved that it doesn’t even need to.

You already know what reality is (or do you? - cue existential rabbit hole) and the term augment means “- to make something better/greater by adding to it”. Put these two together and you have a reality that is made better/greater by adding to it. 

For anyone with more technical sensibilities, augmented reality is:
“An interactive experience of a real-world environment, where the objects that reside in the real world are enhanced by computer-generated perceptual information”

Social media filters are one of many examples of the AR-powered services available today.  

Google translate launched in April 2006, using linguistic data from the United Nations and European parliaments. The service came with its shortcomings. 

For instance, It handled simple and short sentences well. But, its translations lacked context and disobeyed the rules of sentence structure as the sentences got longer and more complex.

In 2016, Google began to use a technology called neural machine translation, NMT, a complex machine learning system that analyzes patterns in language, based on large quantities of documents. This technology was powered by strides in machine learning (a subset of artificial intelligence), and it was programmed to imitate humans that were learning a new language for the first time. The new system not only improved translation but Google’s speech and image recognition capabilities. 

It is important to note that augmented reality doesn’t do the actual job of language translation. That’s the work of Artificial intelligence - as I briefly described. Instead, AR pulls its weight in the areas of service delivery and user experience.

Take Word Lens for example. Released in 2010, this application used optical character recognition (OCR) for language translation. In real-world use, you would take an image of the text you wanted a translation for, and your camera would scan the image and identify the characters as it did for the lady in the video that follows. 

When Google integrated the ‘word lens’ feature into the Google translate mobile app in 2015, the feature translated printed text in real-time.

This is a clear example of how augmented reality pushes for more seamless integration of innovations across the board.

In 2022, emerging AR capabilities are taking language translation a step further.

At this year’s Google I/O conference the company announced a prototype for AR glasses. Google CEO, Sundar Pichai, described the glasses as a piece of hardware that deliver the output of Google’s advancements in translation and transcription to your line of sight in real-time.

This means that with Google’s AR glasses on, you would be able to communicate, in real-time, with someone that spoke an entirely different language. This is possible because the glasses help you ‘see’ what the other person is saying - their words appear as lines of text in your natural line of sight. 

Product manager, Max Spear, calls it “subtitles for the world”.  You can see why in the following video. 

Augmented reality shines when our relationship with technology becomes more intuitive. The goal is always for the technology in itself to fade into the background, to be taken for granted even - that’s when you know a product has fully integrated with everyday life. And AR looks to explore innovative ways to hide in plain sight. 

Earlier, I casually defined augmented reality as a reality made better/greater by adding to it. And while still only a prototype, Google’s AR glasses confirm this definition. If/when they hit the open market, they just might revolutionize communication. That’s a better reality if you ask me.  

Who knows? I might finally stop feeling pangs of guilt every time I ignore my duo lingo app.