The Present and Future of Augmented and Virtual Reality

In September Apple and Google released ARKit and ARCore — two tools for developers to push forward the creation of augmented reality (“AR”) applications. Besides, the new iPhoneX contains facial recognition technology which is another important step in the field of hardware development which brings us closer to augmented and virtual reality (“VR”) experiences.

Pokemon Go and the explosion of AR

The technology around AR has been developed a lot in the academic space. However, only recently mobile devices have gotten fast and good enough to be able to run some of the very advanced algorithms. In other words, AR requires powerful hardware and software to support advanced computer vision work.

Pokemon Go brought the first step of AR in terms of quick mobile experiences, it made it easy for people to go out and experience augmented reality. The success of Pokemon Go could be explained by two factors: (i) taking some of the most advanced technology and a new type of gameplay and (ii) combining that with a very well known and nostalgic brand. Those two factors helped Pokemon Go to reach the masses and help the society understand what augmented reality actually looks like.

Besides, Pokemon Go encouraged people to get out of their homes and play that game together in the real world. This was almost like a dream come true: Pokemon Go created a virtual world that looked like real world and made us believe that Pokemon exists. Hence, getting that shared experience of having Pokemon around us as well as adding some exercise-related activities played a major role in the success of this game.

The Industry’s Push towards AR

Two tech leaders of the world, Apple and Google, launched ARKit and ARCore in October. These two systems mark a significant push towards the democratization of AR experiences. ARKit and ARCore can run on millions of mobile devices. The API and SDK are relatively simple to use and allows people to experiment in the ways they have never experimented before.

Yet, at least at the outset, ARKit and ARCore are going to experience some limitations. First, ARKit and ARCore offer only a phone experience which face the problem of tracking: overlaying and doing a six degrees of freedom (6DoF). This only covers the front end (i.e., visualizations). Once developers start building more unique experiences, it will be necessary to build backend which is essentially different from backend solutions in the web.

Second, if users want to share their AR experience with multiple people and have a shared experience of multi-player AR, this is not (yet) be possible with ARKit or ARCore.

Expect the Unexpected

Soon we will get into a phase in which AR will become very playful. This form of experimentation will lead to new great applications of AR. At the moment it is rather difficult to predict what kinds of applications we will see.

A great example of that unexpected experience is Instagram: in the beginning, many were asking whether a photo app with filters is necessary at all and instagram was undersold. Yet, now Instagram is one of the most popular social apps and one of the most valuable properties of Facebook.

Another analogy could be iPhone itself. In 2007–2008, when first iPhones were launched, we saw how very apps helped using flashlights, and that it took quite a bit of time for more exciting apps to appear.

Similarly in the field of AR: we will definitely see solutions that have extremely high viral growth. Meanwhile, there will be a phase of learning and experimenting mainly by artists and software designers in trying to figure out what can be built.

At present, we are in the stage of past-genesis where building of AR apps has just begun. In the coming year or two there will certainly be a lot of trash we will see developed. We should also have in mind that usually it takes about 9 to 12 months to develop a great app.

ScopeAR solution

Without any doubt, great AR apps are currently in the process of making and soon we are going to see them. It is likely that some of the amazing AR applications will firstly appear in the area of enterprise applications. Those apps will make learning and training easier. One of such AR learning tools is provided by ScopeAR.

The great thing about enterprise AR solutions is that the problem range is relatively narrow. This makes it easier to make an application which is limited in a prescribed space or other factors that could affect the development.

Factors affecting the boom of AR apps

Firstly, there are two new things that make AR unique: (i) AR will make the positioning of the device more important; and (ii) the new AR content will offer mobile device users totally new experiences.

Once developers begin to understand those two key factors, then we will start seeing the boom of AR. Uber could be a good learning example: it was not possible to make an uber style of app in the early ages of PCs; uber only starts making sense when the mobile platform comes out.

Further, also the whole canvas of creating and AR app is different from traditional mobile apps. Until now we have been accustomed to see screens of our mobile phones as a canvas. With AR, all surrounding space around us becomes a canvas upon which we can experience AR. The change in perceiving the canvas will also lead to dramatic changes in user experiences.

AR will also change the perception of the location of the user of the mobile device: rather than interacting with their phone, AR will empower device users to start interacting with the space they are in.

Simultaneous augmentation

Together with AR, we can expect new attempts to provide a simultaneous augmentation of audio, geo-spatial and visual experience thus leading to an experience which changes the perception of the surrounding world. This might be an interesting transition from the world we live in now towards an augmented world.

What stops us from getting to augmented reality? The biggest factor here is rime_._ As mentioned before, first of all developers and artists will need to understand the potential of augmentation; and once this initial period passes, we will see the next stage of AR.

In October 2017, Apple has released its new iPhone X which contains a breakthrough facial recognition technology. This facial recognition technology has become possible by combining four components: (i) ordinary camera, (ii) infrared camera; (iii) dot projector; and (iv) flood illuminator.

The AR/VR industry has hoped though the same technology to be introduced on the backside of iPhoneX, which could have been even a greater breakthrough in the development of hardware and could have opened even further potential for AR/VR development.

Yet, most of the new generation smartphones already have capabilities to recognize surfaces which helps to place AR objects and introduce various AR experiences. But developers are thinking further. For instance, the creators of CurioPets, an app which enables mobile device users to have their own digital pet, think about using iPhoneX’s facial recognition features to add some personalty features to CurioPets. Namely, depending on the facial expression of the iPhoneX owner, the pet should be able to react accordingly.

“It is really important to connect emotional aspect to the use of AR technology.” (Nathan Kong, CEO of CurioPets).

Changing patters of using mobile devices

The phase where AR reaches mobile phones and enables users to have augmented world experiences will certainly last for a while. However, we may wonder whether people will be willing to adapt to the new ways of using hardware (holding their devices in certain positions as necessary for the AR apps to function). Certainly, not all people will be willing to adjust and change the habits they use their devices.

AR developers are currently pondering what are those ways to facilitate the change in human behavior. For example, an average Snapchat user uses the app for 15 seconds at a time; mobile AR experiences are expected to capture 2–5, in some cases up to 15 minutes, of user engagement.

So one of the underlying questions is what could make the AR technology compelling enough? We are certainly in the early development cycle which makes it very hard to predict.

One of the primary applications of AR are sporting events and concerts. The developers will piggy-back on current habits of spectators who are keen to take a photo or even live stream an event they are taking part in. Therefore, it is not hard to imagine how concerts or sporting events could be augmented: showing statistics of players or having imaginary dragons emerge from behind the stage (example: SKT baseball stadium in Korea).

Current challenges of AR

There are three main challenges for the further development of AR: power/ battery life, optics, and fashion_._

Optics require new discoveries in the area of physics. So optics pushes the boundaries of research which also depends on luck. Optics is even more so challenging in outside environments where there is a huge photon generator up in the sky :)

Optics is a hard nut because of existing technologies. Engineers have to reconsider how the image comes to a human eye and find further solutions related to augmentation of the surrounding world.

Fashion solutions also matter a lot for the adoption of AR gadgets. If we are thinking about eyewear or glasses, they should be pretty cool, otherwise there will be nobody who wants them. At present we are quite far away until somebody is going to make small glasses of quality.

Some experts predict that we are around five years away from shrinking down the technology and producing something that would be widely adopted. Also, billions of dollars will have to be spent to make that next amazing wearable device.

Still, one exciting thing about the AR is that it requires totally new backend architectural as well as UX solutions; even though it may take some time.

Is it possible to speed up the development cycle of AR/VR? Technical issues posed by the AR are of a very different nature than in other technologies. The same problem applies to headsets. Software engineers will have to learn from the real world and also develop new programming patterns that fit AR space. Currently, mobile devices pose two-dimensional problems. However, AR/VR brings about three-dimensional problems that have not been seen until now.

AR opens new markets and new spaces to create content and offer totally new experience for all of us. Those who will manage to get those nuggets related to augmentation will be able to collect the payoff.

Exciting times! :)

Thank you for reading this post! If you liked it, click and hold the 👏 on your left side, or leave a comment.

I publish a a new story every week. Follow me and you won’t miss my latest insights on innovation, creativity, and the recent trends in Silicon Valley and beyond!