Last night a coworker of mine, Sonam Dhingra, gave a great talk about some of the basics of ARKit, and touched on some of the more advanced topics like touch handling on 3D objects and blend shapes. Having done AR in production for the past year at Nike (see https://9to5mac.com/2017/06/23/nike-snkrs-ios-app-augmented-reality-exclusive/ and https://www.digitaltrends.com/outdoors/inside-s23nyc-the-digital-studio-keeping-nikes-tech-relevant/), a lot of people came up to us last night after the talk to ask about some of the pain points in working with ARKit and SceneKit and whether a 3D engine like Unity is better suited for their purposes. I made a mental note of some of those questions and decided to write about how I reach a decision about which framework to use.
What they are, and what they do
To make an informed decision, let’s understand what each of the two frameworks do and what their strengths and weaknesses are. Note: I am choosing to skip over Unreal engine because in its current form, it’s not as indie-friendly as Unity, and the ecosystem is not as approachable. If you are arriving at a decision as an organization, you should look at Unreal as well.
Unity: A full 3D engine, including an IDE which has a scene editor (allowing you to place your objects and create your world), physics engine, animation editor, full GUI for editing object properties, attaching scripts to your objects (behaviors), material editor, lighting manager (so you can bake your lighting into light maps), and a host of other assistant editors.
The script editing is done via an external application (Visual Studio Code is the default solution), and the language used to develop is C#. The produced output is cross-platform (i.e. games made in Unity can be easily ported to around a dozen platforms, including iOS, Android, macOS, Windows, Xbox, Playstation, Switch, etc.)
ARKit/SceneKit: A mostly barebones 3D engine, with minimal editor support through Xcode. The editor allows you to lay out your scene, and has a properties editor for your objects, allowing you to change basic properties like position, rotation, scale, and visibility features, including a materials properties editor. There is very basic support for adding SCNActions such as rotate and scale animations in a timeline. There is a lighting tab which allows you to bake your lighting, but has no real properties to tweak how that occurs.
All your code is in native Swift (or Objective-C if you’re still doing that), and you have to manually set up the boilerplate code to load up your scene, and perform whatever actions you want on your scene objects. You define your own architecture for writing your application logic.
What do I use?
The question that came up most often last night was, “Which platform should I use?” While not always cut and dry or obvious, there are a few rules of thumb that I’ve used to determine which platform to use for a particular use case.
Let’s go over some of these scenarios and see which platform I would choose.
- You are prototyping an idea
- You are building a consumer app (non-game) that uses standard UI elements (UIKit standard elements like UIViews, UIButtons, UICollectionViews, etc)
- You are building a game
- You are adding AR capabilities to an already existing application
Let’s examine each of these.
You are prototyping an idea
When prototyping an idea or determining viability of a certain use case, the overriding factor is how quickly can you get a demo version up and running. Being able to test out an idea quickly is essential in figuring out whether or not to spend weeks or months further developing that idea into a full-fledged product.
You can’t spend weeks on a prototype so you need to choose the platform which will allow you to get to a workable demo quickly. That means spending less time on writing setup boilerplate code like object loading and animation and more time on getting your idea to the screen. UI is less important in this case, and a minimal interface is fine for situations like this.
In this scenario, Unity wins easily. I can create a new project, drag and drop some basic (or not so basic) models into my scene, create a new script and be coding a behavior for them in under 5 minutes.
Using SceneKit, it’s a lot more complicated. Even using the Xcode template for ARKit or SceneKit still requires you to write loading code, object management code, and start keeping references for your objects so you can then write code to manipulate those objects. If what you’re looking to load is anything more complicated than a simple 3D model, you may encounter importing issues (I’ve encountered several issues with SceneKit’s Collada DAE format loading) which will lead you down a rabbit hole of going into your 3D modeling application (Cinema4D, Blender, etc.), tweaking, exporting, re-importing, and testing.
In this scenario, I would hands-down choose Unity. Even if you set up boilerplate code to initialize your scene and develop some sort of framework to mimic Unity’s object behavior pipeline, it is still a lot of work to get an idea up and running. The only time I choose to go the SceneKit route for prototyping is when I am experimenting with a particular idea that I want to determine the feasibility of doing in SceneKit. That is, I want to know if it’s possible to do this particular thing using SceneKit’s API. It does me no good to figure out how to do something in Unity if in the end it can’t be done in SceneKit, or I would have to learn how to do it all over again.
You are building a non-game app
If you already know what you want to build, and are ready to begin writing your app in earnest, then you need to determine which is the best platform for the type of app you are looking to write. If you want to have your app look native to iOS, for instance, using native UI controls, or using built in mapping, notifications or web views, then I would choose to go native and use ARKit/SceneKit.
To be able to use native UI in Unity is another rabbit hole that I urge people not to do. You can use native UI over Unity, but it involves building bridges to the native side using Unity’s plugin architecture. I can attest from experience that it is not a pleasant task, and you end up in a sort of uncanny valley where you are mixing native UI with in-experience (3D) UI and it just does not look good.
There are off-the-shelf plugins you can buy in the Unity Asset Store (more on that later) that you can use for mapping or web views or other native interfaces, but there is usually a cost involved, and most of the time it’s not just money, but time and effort in dealing with the integration of 3rd party code.
In this use case, I would choose to go native and use SceneKit.
You are building a game
This is probably the easiest scenario. Even though currently there are only two platforms that support mobile phone AR (iOS with ARKit and Android with ARCore; I’m leaving Vuforia out for this argument), Unity provides the best chance of staying cross-platform. It is built with games in mind, and has all the necessary components at the ready. Physics, collisions, animation, input schemes are all Unity’s strengths. The Asset Store provides additional resources in the form of plugins, 3D model assets, GUI packs, audio effects, music, etc. It is a mature ecosystem now, with a plethora of free and paid assets. Want to build a MOBA-style game where your characters have to traverse your office in AR? There’s an asset pack for that. Need multiplayer networking to take your game to the next level? There’s plugins for that too. Need a fancy race car or samurai or jet plane or mechwarrior? There’s plenty to choose from.
The amount of work needed to replicate any of the things that Unity provides in the case of using SceneKit is tremendous. Only if you already have all the necessary underlying code to do all the above things really well would I even consider using SceneKit for a game. If you take out the AR component altogether, that is, if you were building a 3D game, there is no way I would even consider SceneKit.
In my opinion. there is really no choice to be made here. It is obvious I would choose Unity.
You are adding AR to an already existing app
This is the choice we had to make at Nike. We had to make a decision on how we would add AR capabilities to our already existing application, the iOS version of SNKRS (iOS and Android versions here). This is an already-established app in the App Store, with rich media and a lot of functionality. It has a very distinct design and follows strict style guidelines.
Trying to integrate a full engine like Unity into SNKRS would have meant integrating the whole Unity runtime, writing hooks to propagate touch events and interactions, writing plugins for things like location and network access (so we would make use of our own networking stack), and figuring out how to overlay our own UI on top of the Unity view, all while attempting to steer clear of the uncanny valley. For us, in this situation, it didnt make sense.
Our use case at the time was relatively simple, and we didnt need the overhead of a full 3D engine and all its associated (and unused) functionality that comes with it. Not only that, but the addition of the runtime & scene would have added upwards of 30MB to our binary. Not something that we could do.
For us, the choice was clear. Integrating SceneKit into our app was as easy as just adding another view controller into our app, just like we do for anything else. Beyond the setup code, all of the other functionality involved loading assets from the network, displaying them on the screen, and using well-known conventions like gestures and using native UI. The result is a seamless experience that looks natural. No uncanny valley here.
No question about it here. SceneKit was the clear choice.
The elephant in the room
There are a lot of factors in choosing one technology over the other, but a deciding factor can often be “How do I get the same functionality on both iOS and Android”. While the technology and interface to ARCore and ARKit differ, your logic for controlling your scene can remain the same. This is where Unity shines. One codebase, multiple platforms. If your app is a game or has non-standard UI, then Unity may be the right choice, particularly if you are short on developers and their strength is C#. You could write two different bridges to the ARKit and ARCore APIs, then all your logic is separated and can work with whatever AR engine happens to exist on the current platform.
Other Comparisons
3D model loading
SceneKit falls short here. While it can support other formats like Alembic’s ABC and Wavefront OBJ’s (see https://developer.apple.com/documentation/modelio and https://developer.apple.com/documentation/scenekit/scnscenesource), in my opinion the only real option is Collada DAE format, since it can hold multiple objects, material descriptions, and animations. I never had much luck importing a model in Alembic format, but that could be due to bad exports from 3D modeling software. For me, that’s reason enough not to use it. There are external libraries like Assimp and AssimpKit (an iOS wrapper) to assist in loading other formats, but they add complexity to your project. Unity wins easily here, supporting a lot of formats, including the popular FBX format, and better support for importing improperly exported files (it seems every 3D modeling package has its own interpretation of how to save files in different formats).
Plug-ins and Assets
If you are looking for 3rd party libraries for things like networking or custom UI components, both sides (native iOS and Unity) have a rich ecosystem of Cocoapods and plug-ins, respectively. If you are looking for game specific assets, such as 3D objects and audio or music, then the Unity Asset Store can fullfill your needs, while you are looking at purchasing assets on Sketchfab or TurboSquid on the native side. I’ve found that it’s usually cheaper for the same items on the Unity Asset Store due to the increased competition. YMMV.
Documentation and Sample Code
Putting aside Swift and C# documentation and sample code, which there is an abundance of, and concentrating solely on SceneKit and Unity API’s, the lack of SceneKit documentation and sample code is alarming. When I first began to dabble in AR and wanted to do something more involved than putting a cube on the screen, things got hairy. Want to write a custom shader or learn how to modify a group of vertices? There’s almost no docs or sample code to show you how to do that in SceneKit. I’m attempting to help people get started by writing more about ARKit and SceneKit. Getting started with ARKit and SceneKit, ARKit, SceneKit, and How to Control the World, and an upcoming post try to close that gap. In Unity land, there’s tons of sample code, demo projects and a large community of people doing amazing things. SceneKit’s been around for a few years, but there’s never been much of a reason to use it, thus, a lack of resources for it. With ARKit now making heavy use of it, perhaps we will see a surge of new materials being available.
3D Scene Layout and Object properties
This is Unity’s strength. They have a fully-featured 3D scene editor, and now, with the purchase of ProBuilder, they have a way of creating geometry within the scene editor. Their property editor allows you to drag and drop scripts (behaviors) onto objects and be able to edit script properties (values, lists, references) all within the editor. SceneKit is severely lacking in the editor department. It has rudimentary control of object layout and transform properties. It allows you to assign material properties, and that’s about it. It has an animation timeline that let’s you drop SCNActions to it and have some simple transform animations, but beyond that its capabilities are limited.
Conclusion
The above guidelines are what I use to make my own decisions when choosing a framework at the start of a project. Your own situation and knowledge is probably different than mine, and your final decision may differ. If you are an iOS developer and have never used Unity, or feel intimidated, I urge you to check it out with an open mind. It has its positives and negatives, but having options never hurts.
If you enjoyed this analysis, please recommend and share the story with others. Do you have your own experiences with SceneKit or Unity? Comment below and share your thoughts.
Daniel Wyszynski is a developer who’s worked on more platforms and languages than he cares to admit to. He believes in building products that create memorable user experiences. To Dan, the users are first, the platforms are second. Follow Dan on Medium or Twitter to hear more from him. Also check out the s23NYC: Engineering blog, where a lot of great content by Nike’s Digital Innovation team gets posted.
Author’s Note: Views are my own and do not necessarily represent those of my employer