Creating the right products for VR, AR, or MR

Note, opinions are my own and do not reflect my company.

Several articles have thoroughly explained the difference between virtual reality (VR), augmented reality (AR) and mixed reality (MR); however, few articles have described the best types of products for each platform. In short, virtual reality closes off the real world, and completely immerses you with the feeling of presence in a digital world. Augmented reality is when you simply overlay digital content on the real world. For mixed reality, my friend Graeme Devine said it best: mixed reality is the mixture of the real and virtual worlds, so that one understands the other. In mixed reality, you feel copresence, such that you believe digital content is in the real world with you.

With the rise of the various VR platforms like the HTC Vive and Oculus Rift, and AR platforms like ARKit, more and more developers are exploring potential products across all of these mediums in the hopes that one of these platforms takes off in the future. You can see a sample breakdown of what platforms developers are currently exploring in a recent report from the 2017 Virtual Reality Developers Conference (VRDC) where 600 developers were surveyed.

However, to build a successful product for one of these mediums, developers need to truly understand what differentiates them.

• Virtual Reality. One unique strength of virtual reality is the large Field of View (FOV) — by completely closing out the real world, you can provide unparalleled immersion and presence for users, which is perfect for taking users to fantasy worlds such as Gringotts Wizarding Bank from the Harry Potter Universe, or remote but real locations such as the planet Mars. However, one of VR’s core weaknesses is locomotion (the inability to move around naturally without requiring a large space that no average user in the Valley ever has). Many products have tried to design around this problem with teleportation mechanics, where you can point at a location then tap a button to teleport there; some companies like NeatCorp have even done a great job optimizing this design to conclude that projectile mechanics results in more accurate teleportation for users. Even with this design, the fact remains that locomotion is not a strength of virtual reality — you have to design around it for now. So until the technology improves, developers should avoid directly translating game genres such as First- or Third Person Action Platformers (like Mirror’s Edge) or “Run and Gun” First Person Shooters (like Quake) that require precise, skill-based locomotion. Instead, developers should focus on products that take full advantage of the strengths of VR, such as On-Rail Experiences (where an experience intentionally has no locomotion and is “on rails,” such as being a passenger in a moving vehicle in the award-winning Pearl, or participating as a passenger in a car chase where the user has to look around and shoot at targets). Another great genre of games for VR right now are “cockpit” games such as Elite: Dangerous VR where users play a pilot in the cockpit of a vehicle, and the entire game is played while the character is stationary.
• Augmented Reality. I am going to keep this section brief, because depending who you talk to, it is a either a sub-category of mixed reality, or the same thing. One of the core strengths of augmented reality right now is the ability to be mobile with normal locomotion. You can experience augmented reality wherever you go, and some games like Pokemon Go have explicitly designed their mechanics to encourage exploration of the real world. However, many augmented reality platforms currently lack the ability for users to have complex interactions with the digital content, and are contained to small field of views (FOV) and screens (such as that of a mobile phone or tablet).
• Mixed Reality. AR and MR are probably the least understood mediums of the three. One of the strengths of MR is the ability to feel copresence, and bring digital content into the real world without excluding yourself from those around you. Although it feels magical to simply see digital content interacting with the real world, such as a digital character walking around furniture in your living room, it is important to realize this is heavily driven by a psychological phenomenon known as the novelty effect. For many people, content created by something like ARKit is the first time they have seen a digital object in the real world, and that alone is exciting. However, if you design products or games that rely heavily on the “wow” factor of seeing digital content in the real world for the first time, your product will fail. As more and more MR products release, or as users experience your product repeatedly, they will start getting accustomed to seeing digital content in the real world, and what used to be a “wow” will soon become “so what?” This also raises the importance of proper user testing. If you are currently creating an AR/MR product, the novelty effect is so powerful that every user that steps into your offices to test your product will most likely give you glowing results; however, that is a red herring. For current AR/MR products, it is critical to do user testing focused on repeated sessions — get the same user to try your product an hour a day for multiple days, weeks, or months. For now, this is the only way to get accurate feedback for AR/MR products.

So what should developers focus on in augmented or mixed reality?

• Core Loops. A popular term among designers in the Video Games industry, the core loop is what makes something engaging. It is the single most important part of any game or product, and is an intentionally designed chain of actions a user takes in a product (and is hopefully fun to repeat!) Because the novelty effect is so powerful in AR/MR these days, it is important that developers ignore the initial “wow” of seeing digital content in the real world, and focus on the basics of product design and develop a strong, engaging, fun core loop.
• Interactions. When you put digital content in the real world in front of someone, the vast majority of users want to interact with the digital content by reaching out their hands and simply grabbing the content. Many developers have focused on using a console controller, remote controller, or other input device that is an abstract mapping of controls for the digital content in the real world, and in many cases, it works decently. However, the developers that create products that allow users to do what they want and reach out and interact with the digital content in more physical ways will ultimately win. Secondly, it is critical right now to focus on experiences where digital content is interacting with the real world — without this, it simply is not mixed reality.
• Less is More. Because putting digital content in the real world is so amazing, a lot of developers erroneously start filling the entire world with digital content.

This is not a world any of us want to live in. One, it basically overwhelms the user with a sub-par VR experience, and no longer feels like AR/MR. Two, users are so used to the concept of a “digital screen,” that when this much digital content fills your field of view (FOV), they began to treat the experience like a screen and stop moving around (one of the strengths of AR!)

Hopefully, this article has helped your product team better understand one strength and one weakness of virtual reality (VR), augmented reality (AR), and mixed reality (MR) so that your team can make the best product for your chosen medium. When creating a product for these mediums right now, always ask yourself, “Is the current product design better in one of the other mediums?” If so, you may want to pivot sooner than later.

ABOUT THE AUTHOR:

Jeffrey Lin, Ph.D.

Dr. Jeffrey Lin was a Lead Product Owner and Lead Designer of the award-winning PC game League of Legends at Riot Games, one of Fortune’s Best Companies to Work For. He was also a Research Scientist at Valve Software, makers of the award-winning PC game Portal 2, and creators of the Steam platform. He obtained his PhD in Cognitive Neuroscience from the University of Washington where he was funded by the Howard Hughes Medical Institute. His design work has been featured in Wired Magazine, MIT Tech Review, The Verge, Scientific American, Times Health & Science, and Re/code. His research has been featured in numerous peer reviewed journals, including Nature.