What is Apple ARkit and how to get started with it?

This year at Apple’s Worldwide Developer Conference (WWDC) in San Jose, California Apple unveiled “ARKit”, a new, free set of software tools that anyone can use to build impressive new apps and experiences on the iPhone and iPad. The “AR” in “ARKit” refers to “augmented reality”, a technology that places interactive virtual objects and effects into your view of the real world, blending the virtual world with physical reality. iOS 11 introduces ARKit that takes apps beyond  the screen, freeing them to interact with the real world in entirely new ways.

Features of ARkit as described by Apple

• ARKit uses Visual Inertial Odometry (VIO) to accurately track the world around it. VIO fuses camera sensor data with CoreMotion data. These two inputs allow the device to sense how it moves within a room with a high degree of accuracy, and without any additional calibration.
• With ARKit, iPhone and iPad can analyze the scene presented by the camera view and find horizontal planes in the room. ARKit can detect horizontal planes like tables and floors, and can track and place objects on smaller feature points as well. ARKit also makes use of the camera sensor to estimate the total amount of light available in a scene and applies the correct amount of lighting to virtual objects.
• ARKit runs on the Apple A9 and A10 processors. These processors deliver breakthrough performance that enables fast scene understanding and lets you build detailed and compelling virtual content on top of real-world scenes. You can take advantage of the optimizations for ARKit in Metal, SceneKit, and third-party tools like Unity and Unreal Engine.

Examples of Apps with AR features

There are apps that have AR features and use smartphone such as iPhone camera to show you a view of the real world in front of you, then layering virtual objects on top of it. Like some of the most popularly used AR features right now are the “lenses” or face filters in Snapchat,  the effects that put animal ears on your head or warp your face in weird ways. Another one is Pokemon GO, the gaming app that made the pocket monsters look as though they were bouncing around your desk or the street. What these apps have in common is that the virtual objects inside of them appear to be inhabiting or changing real-world scenes — “augmenting” our reality.

Apple’s ARKit version of augmented reality so exciting is that it isn’t an app  instead, it’s a free way for anybody to build new AR apps that will be able to run on many of Apple’s iOS devices from the last few years. Apple’s ARKit has already only been publicly available, but already, independent software developers have created some incredible-looking experiences for the iPhone — everything from virtual portals you can enter to putting virtual zombies in your living room. Even Lord of the Ringsdirector Peter Jackson has gotten in on the fun with an ARKit short film.

How to get started with ARKit

To get started first check to make sure you have a compatible iOS device. ARKit runs only Apple iOS devices containing an A9 processor (or newer). That limits it to the following devices, according to RedmondPie:

• iPhone 6s and 6s Plus
• iPhone 7 and 7 Plus
• iPhone SE
• iPad Pro (9.7, 10.5 or 12.9)
• iPad (2017)

Also its is assumed that the new iPhone 8 lineup will also support ARKit. You’ll also want to make sure the camera on your iOS device is working, as ARKit relies upon it. Also you need a cable for connecting your device to your Mac computer.

Once you have one of these compatible iOS devices handy, you’ll need to download iOS 11 if you have older device and not updated your iOS

You’ll also need a computer for coding your ARKit app. Specifically, a Mac with the free operating system mac OS Sierra 10.12.4 (or newer version) installed. That means the following machines are eligible:

• MacBook (Late 2009 or newer)
• MacBook Pro (Mid 2010 or newer)
• MacBook Air (Late 2010 or newer)
• Mac mini (Mid 2010 or newer)
• iMac (Late 2009 or newer)
• Mac Pro (Mid 2010 or newer)

Next register for an Apple developer account. You can use a free or paid developer account to get started with ARKit. If you already have a regular consumer Apple account that you use to buy apps and such on your iPhone/iPad/Mac, you can upgrade this to a free or paid developer account. Using a web browser on your Mac computer, navigate to Apple’s developer account sign-in page and choose which option is right for you.

On your Mac, sign into your Apple developer account, then head here and click “Download”in the top right corner to get Xcode 9, Apple’s program for programming also note you’ll need at least 4.9 GB free of hard drive space to download this, and then Xcode itself takes up 9.76 GB of hard drive space.

Now launch Xcode 9 and connect your iOS device to your computer with a cable. All you need to do is unzip your Xcode 9 download by double clicking it, then drag the Xcode app into your Applications folder and double click it to launch it. Plug in the iOS device you have with iOS 11 installed.

To create a new AR app project template in Xcode on your Mac, click “File”, “New”, and select “Project”. This should pull up a grid of options. Click on “Augmented Reality App”.

Next fill out your new AR app’s information in Xcode as Apple requires you fill out the following information for your new AR app. You can name it anything you’d like, and be sure to also select a “Team”, even if you’re an individual (if you don’t have one yet, you’ll get the option to create one). Keep everything else checked.

In the “language” option, can also choose between Swift, Apple’s new programming language, and Objective-C, its older one. Both will allow you to build an AR app. If you’re new to programming, Swift might be the better bet as it’s what Apple plans to use for all of its tools going forward.

Finally, you can choose your “Content technology”, selecting between SceneKit, SpriteKit and Metal. These are different rendering engines for displaying graphics. All three will let you build AR apps.

You’ll also need to put your project in a folder on your computer. Select anywhere you want, this doesn’t matter much. Select your connected iOS device to receive your new AR app

In Xcode on your Mac, select “Window” from the top menu bar, click “Devices and Simulators”, and double click your iOS device. Make sure you check the box that says “Show as run destination”.

You may get a pop up on your iOS device now saying “Untrusted Developer” and giving you some instructions to go to your iOS device’s “Settings” page to allow this app.

If this happens, go ahead and find your “Settings” app on your iOS device, tap it, scroll down to “General” and tap it, then scroll further to “Profiles and Device Management”, and you should be able to find your Apple developer email address. Tap this and it should allow you to install your ARapp.

Now run your first demo AR app as Apple includes a very simple AR demo with Xcode for you to get a sense of what you can build with ARKit.

In order to see it in action, first make sure your iOS device is selected in the device picker dropdown menu in the top left hand corner of Xcode. Then, look up at the top menu bar in Xcode. Find “Product”, click it, and then click “Run”. The top status indicator in Xcode should begin flickering with activity and messages, then it should launch your app on your iOS device. To stop your demo app, go back to “Product” in Xcode and scroll down to “Stop”. Similarly try creating more of AR apps and gain experience in using ARKit.