Playing Videos in Augmented Reality Using ARKit

Augmented Reality TV

One of the things I am really excited about is the idea of watching videos in an AR Flat Screen TV. Obviously, to create a seamless experience you need to be wearing smart glasses but it is good to know that watching videos in AR world is possible.

Create an Augmented Reality project in Xcode and register the tap gesture as shown below:

The point of registering the tap gesture is that we want to add a flat screen AR tv to the location where the user has touched the screen.

The complete implementation of the tapped function is shown below:

That is a lot of code here, so let’s go through it step by step.

First we get the current frame from the scene view session instance. The current frame holds the information related to the captured image as we experience it from the camera. This also includes orientation, scale and position of the image.

Next, we create the SKVideoNode instance which is capable of displaying videos. The SKVideoNode is part of the SpriteKit framework, hence the SK prefix. The file name “big_buck_bunny.mp4” is part of the project. You can also use a URL to point to the video file instead of making it part of your project. The play function on the video node will make sure that the video is automatically played when rendered on the screen.

Next, we create the SpriteKit scene because SKVideoNode can only be added to the SpriteKit scene. We adjust the video node to be the same size as the SpriteKit scene.

After that we create the SCNPlane object which is a 2D plane in the SceneKit world. The magic happens with the help of diffuse.contents property of the plane where we assign the SceneKit scene. This means the SceneKit scene which contains the video node becomes the contents of the plane material.

After that we simply adjust and make sure our plane orientation is correct and that’s it!

The live demo is shown below:

Displaying WebView Using ARKit

You can also display a WebView instead of the SKScene as shown in the code below:

And here is the result:

I hope you have enjoyed the post. If you want to learn more about ARKit Programming then check out my 8+ hour course below:

Mastering ARKit for iOS | Udemy_Build Amazing Augmented Reality Apps for iOS_www.udemy.com

Happy coding!