9 Must Have Skills To Become An AR/VR Developer (With Course Recommendations)

XR encompasses augmented, virtual, and mixed reality technologies. This is an up-and-coming industry that may be good to get into. An XR Enthusiast can be of 2 types — 

1. Developer
2. Designer

It depends on the enthusiast’s thought process.

This article lists some of the skills that will help you grow in XR, as well as serve as an answer for “How do I become an XR Developer?”

Like many other technologies, there is no perfect roadmap to learning XR. Nevertheless, I have tried my level best to come up with a generalized learning process for the same. 

Save 1000+ hours on AR/VR research. Get my 100+ AR/VR Resources Pack now!

Skill 1 —  Programming

Basic programming experience in languages like Java, C#, Swift, Javascript etc, is essential for developing any piece of software. Get a good grasp on any one of these languages. Concepts like loops, control-logic, OOP principles like inheritance, abstraction, encapsulation, etc, should be at your fingertips before moving forward.

If you already have a bachelors degree in Computer Science, you have a good foundation. If you don’t, then who cares? Learn programming from YouTube, Udemy, Udacity, and a thousand other free platforms.

Skill 2 — Software Development

Once you are through with programming, your thinking will automatically shift towards building stuff rather than practicing problems. 

There are 3 types of Software Development that are pretty mainstream — 

1. Web Development
2. Mobile App Development 
3. Game Development

You should have a basic idea of at least one of them. There are tons of online resources to learn these. 

Note that once you pick a domain, you should stick to it for 6 months at least. That’s 6 months of committed learning. 

There is no use of all that knowledge that you have accumulated in the past 6 months if you don’t apply it to a real world project. If you want to turn this into a career, you should spend an additional 6 months applying your learning in a real world project through an internship or a job.

The more time you spend working on a real world project under an organization, the more you sharpen your domain knowledge.

Based on your learning pace you can shorten the 6 months learning process + 6 months real world project experience into a shorter period. I am just providing you with a rough estimate.

In these 6 months you are expected to know stuff like — 

1. UI/UX Development based on a given mock. If you are given a design you are able to develop that design using code. Naturally, the code depends on the domain that you have chosen. For web development, you should be familiar with HTML, CSS & JS. Then you can dive into a UI Framework such as React. 
2. Connecting your client side app to some API to show dynamic data using JSON Parsing techniques. 
3. Adding 3rd party libraries so that you don’t have to reinvent the wheel. I am not pinpointing any particular library because that again depends on your domain and your app requirements. 

Also, I am not diving deep into things like code architecture, testing, security and fixing memory leaks in your mobile apps. These skills are required to improve the quality of your app, as well as scaling it up. 

If in the future you are working on an XR application that requires reaching out to a billion users, then you cannot ignore these quality aspects. For developing small AR/VR prototypes, you can compromise on the quality aspect.

The more knowledge you have in your domain the more you can think about how to combine things and think out of the box. This is one of the many ways to flex your creative muscles. 

Skill 3 — General XR Knowledge

Now it is time to explore XR. Get familiar with the basic terminologies of XR. Understand the constraints of this technology. The objective behind this step is to get you familiar with terminologies of XR and become comfortable with the words used loosely in the XR domain.

There are 3 domains that come under the umbrella of XR

• AR i.e Augmented Reality
• VR i.e Virtual Reality
• MR i.e Mixed Reality

Here is a course from the University of Michigan. It covers the introduction to AR/VR/XR Technologies, applications & issues.

Developer Mindset

Most XR developers tend to use a pre-existing SDK to develop customized XR experiences for consumers.

For those who are interested in building custom SDKs and libraries that can help fellow XR developers to develop those customized XR experiences, pay close attention to skill #4!

Skill 4 — XR SDKs

SDKs like ARCore, ARKit, ARFoundation, Vuforia and Wikitude are well renowned for creating AR experiences for Mobile Platforms i.e Android & iOS. Apart from these are dozens of SDKs present. Depending upon your app’s feature requirements and your present code architecture, select an SDK and then continue with the development.

Most of the SDKs are still evolving. Once you start using some SDK, there is a high chance you will fall into problems that need more time investment from you to solve those problems. That’s how the XR landscape is at the moment.

In VR, there is an SDK for each VR headset. The Oculus headsets have the Oculus SDK. The HTC Vive headsets have its own SDK. So again, it depends upon the headset that you are going to use for developing your VR experiences.

This course from the University of Michigan focuses mainly on developing XR apps with WebXR, Unity & Unreal .

Skill 5 — Machine Learning & Computer Vision

XR works on underlying Computer Vision Algorithms that are abstracted away in methods & functions of existing SDKs. There is a lot of research going on here in terms of optimizing plane searching, depth mapping, occlusion features, etc. These features can be incredibly tricky to develop. For some, it may even seem like reinventing the wheel.

Designer Mindset

Those who are interested in developing consumer facing XR applications and want their experiences to look better than others fall under this category. 

It’s the same as saying, “Both A and B are note taking apps. I like App A more than App B because I feel it’s easier to use. I wonder why App B does not have this cool interaction that solves so-and-so problem.” 

Likewise there are various design problems in XR Experiences.

If you are have a mobile development background, it is similar to getting the UI design mocks for your android app from your designer, rather than you designing and developing the screens yourself. 

Skill 6 — UX Design Knowledge

If you are not that interested in making your XR experiences look beautiful or brainstorming on how to solve design problems, then skip to the next skill. 

For the ones who are interested in transforming good XR experiences into great XR experiences by incorporating interactions, studying users and understanding where an XR experience falls short, and studying UX Design first. Understand topics like interaction design principles, usability, design thinking process, storytelling, HCI, accessibility, etc. 

Once you are done with UX Design, think about how you can apply the design principles in XR. 

I have gone into this in much more depth in my ebook - Introduction to UX Design for Augmented Reality Experiences.

Here is another course from the University of Michigan that goes deep into the UX Design process behind XR Experiences. 

Skill 7—3D Modelling & Animation

Tools like Blender, Maya and Cinema4D are great for prototyping experiences with animated 3D Models and environment designs. These prototypes serve as a good starting point for showing a concept to your potential clients. 

Many times, you will find yourself in a position where you are not able to communicate your thought process to someone. If developers start working on something that does not match the requirements, it wastes precious time & energy. This is exactly why a prototype can prove incredibly helpful for the stakeholders of a project. Once the overall design is finalized then you can continue with the rest of the development.

Moreover, 3D models have to be optimized and should have certain file requirements before being added to any XR experience. Not doing so can cause performance issues. For example) Take a look at the file requirements for ARCore.

Check out my Blender compilation video😉

Interested in both XR Design and Development?

Skill 8— Graphics Rendering

Shader programming languages like Metal, HLSL and GLSL help developers to customize the look and feel of the models, animation effects and vfx in your XR experiences. This skill lies at the cross-intersection of design and development.

If you have done a Computer Graphics Course in your bachelors, you are at an advantage here. If not, then, you have a learning curve.

Here is an excellent YouTube playlist to help you, if you have no idea about Shader Development.

Anyone with the ability to think in terms of both Design and Development is a UX Engineer. There are multiple roles where design and development play a vital role. I have seen Creative Technologists, Technical Artists, AR/VR Developer, AR/VR Researcher, etc in many XR job postings. I believe the focus should be more on the skill set and the responsibilities that you get, rather than the name of the role itself. 

Sidenote

There are so many platforms like Lens Studio and SparkAR that allows you to develop XR experiences without code, but there is a limit to the amount of customization. XR Experiences developed using these platforms run on their own platforms (Snapchat and Instagram) rather than your own platform. You won’t be able to embed the XR Experiences developed using these platforms in your own mobile app.

There are already a bunch of drag and drop platforms for Web Development and Mobile App Development. Why do you think developers are still writing code to develop web apps and mobile apps?

Soft Skills

Skill 9 — Social Media

I strongly believe it is good to always stay updated with the upcoming trends in whatever field you are interested in. Your social media feed shows you what you have liked in the past. If you could trick the algorithm to show you only XR stuff by liking, sharing and commenting only about XR content, then you are bound to absorb a lot of information revolving around XR. You get to know about events(which are mostly webinars, due to the pandemic these days!), courses, news, tips from industry experts, jobs, community competitions and much more.

Conclusion

The above skills should be enough to become an XR Designer or an XR Developer. I understand that the learning curve is steep, but if you are passionate about this field and ready to invest your time into it, then go ahead. XR is growing. It might take some time to become stable for end users, but as an technologist you can surely invest your weekends into XR.

Save yourself 1000+ hours of research. Get my 100+ AR/VR Resources Pack now.

Also published at https://medium.com/@knightcube/9-must-have-skills-to-become-an-ar-vr-developer-with-course-recommendations-6f18b9b18b94