How I spent a week at Microsoft’s Mixed Reality Airlift

I spent a week at Microsoft’s Redmond HQ attending an MR Airlift hackathon where we were placed into teams, and tasked with building a mixed reality application leveraging Microsoft’s Windows Mixed Reality and Azure Cognitive Services. Special thanks to Mai Nguyen and Dr. Neil Roodyn for running the Airlift, as well as the Mixed Reality and Azure product teams for their involvement and interest in the development community. I’m grateful for the opportunity.

Also, a big “thank you!” to my teammates — Cindy Reichel, Benjamin Peterson, Doug Fahl, and Jeremy Diamond — for making the experience such a valuable one. The way we all worked together to zero in on our application’s scope, then divided and conquered to achieve our goals, made this such a successful and enjoyable experience.

The Prompt

My team’s prompt was to build a mixed reality application for a warehouse employee using HoloLens and Azure Cognitive Services.

I have to admit, I rolled my eyes when I first read this. “This is what I always see HoloLens being used for. Why couldn’t we do something cool with it?,” was what I was thinking. But I would soon come to realize this was the best prompt we could have been given. Here’s why:

Microsoft makes a distinction which speaks to the untapped potential of mixed reality. Microsoft says there are firstline workers and information workers (I believe this how they classify the second group). The later is a traditional office worker and has been the target of the PC and software industry ever since Doug Engelbart’s demo of productivity software in 1968. The former represents a massive, more than two billion strong, untapped market of workers who do not fit within the traditional definition of an office worker, and as such, haven’t benefited from computing in the same way information workers have.

Mixed reality devices such as HoloLens will enable data to exist in a real-world context, and allow software engineers to optimize a host of firstline worker related operations that have thus far been out of the reach of traditional information worker computing platforms. When you add cognitive services, such as the ones offered by Azure, these technologies represent a truly transformative potential for computing.

In retrospect, my presumptions about warehouse HoloLens applications being played-out and mundane were wrong. In fact, they are quite the opposite.

The Problem

One of my teammates, Cindy Reichel, owns a shop here in Seattle and pointed out that she would love to be able to use HoloLens to keep track of her store’s inventory. She was able to provided the group with real-world data on how resource consuming and error prone it is to keep track of inventory with the tools available to small business owners such as herself.

Thus, the objective of our application would be to optimize the task of taking inventory by making it more intuitive and engaging using HoloLens and Azure Cognitive Services.

Here’s my team’s final slide deck that establishes our assumption and outlines our solution.

The Demo

Here is Cindy demoing our HoloLens application using a mock storeroom shelf. She uses HoloLens gestures as well as natural language to interact with the application and complete an inventory of our mock retail store backroom.

Reflections

The MR Airlift taught me a great deal about developing for HoloLens and working with Azure Cognitive Services. However, the most important lesson I learned was that you should be thinking outside the potentially narrow borders of your industry, and consider emerging tech as an inter-related whole.

The reason I assumed our HoloLens prompt would be boring was because I only considered the mixed reality aspects of the prompt, and not the potential of incorporating cognitive services.

I went into MR Airlift steadfast in my belief that I was from the AR/VR/MR/etc. industry, but I left as someone interested in the intersection of all emerging technologies, not just one specific technology. I don’t believe there is a meaningful AR/VR/MR industry without AI/ML, chatbots, IoT, and blockchain industries. Likewise, other emerging technologies become more interesting when you consider how AR/VR/MR can be applied to them.

In the end there are just old ways of doing things, and now, thanks to companies like Microsoft, vast amounts of technology at our disposal to optimize those old ways. As problem solvers, we’ve never had it easier. The HoloLens inventory app my team and I created over the course of a week would have been impossible just a handful of years ago.

So let’s keep asking questions, stay curious, and start building!

My official Mixed Reality Developer socks…my Micro-socks…

Let’s keep the discussion going!

I’ve started a vlog series called Inside Out with the intention of exploring the personalities behind emerging technology. Technology doesn’t progress on its own, it is compelled forward and defined by the emotions of the individuals who build it. We can’t fully understand the technology if we don’t understand the people behind it. If this sounds like a discussion you would like to have, hit me up! And if you’re in or around the Seattle area, I would to grab a coffee!