Learn how to build mobile apps with NativeScript using JavaScript, by accessing Apple’s ARKit to add augmented reality functionality.
If you haven’t seen Augmented Reality in action on mobile yet, let me demonstrate with a very practical app that lets me add virtual Christmas trees to my yard.
This app is IKEA Place, and it’s an iOS app built using Apple’s ARKit APIs.
NativeScript lets you access iOS and Android APIs to build mobile apps using JavaScript, and ARKit is no exception. In this article, we’ll look at how you can build your first AR iOS app using NativeScript.
NOTE:
– To run ARKit on iOS you must have an iPhone 6S+, a 5th generation iPad, or an iPad Pro. You can find a full list of supported devices on Apple’s documentation.
– Support for Android’s ARCore in NativeScript is in progress. We prioritized iOS support because Android’s ARCore is still in a “developer preview” state, whereas a stable version of ARKit shipped in iOS 11.
Starting Your App
NativeScript’s AR support is available through a nativescript-ar plugin. The plugin abstracts the ARKit APIs into a set of easy-to-use APIs for common AR tasks. Eventually, those same APIs will also support Android’s ARCore, so that you can build cross-platform AR experiences from a single codebase.
To get started with the plugin, let’s first create a new NativeScript app using the NativeScript CLI.
,
,
Two notes:
Now that you have a new app, you need to use the tns plugin add command to install the nativescript-ar plugin.
,
,
From there, open your app’s app/home/home.component.ts file and paste in the following code.
,
,
We’ll discuss this code in detail momentarily as there’s a lot going on. But first let’s first run the app so you can get an idea of what this example looks like in action.
Unlike most other iOS APIs, you cannot test AR apps using an iOS simulator. So as a next step, plug in your iOS testing device via USB, and run the following command to deploy your app to your device.
,
,
After the build and install finish, the first thing you should see is a prompt for camera permissions, as AR needs camera access to work.
,
,
After you grant permissions, point your device’s camera at a flat surface. After a few seconds you should see a few dots, and then a transparent plane appear. The video below shows what this process looks like.
TIP: Make sure the area you’re testing in has plenty of lighting. If you’re in a dark area, try turning on your device’s flashlight to give the space more light.
,
,
The addBox() method is one of many that the nativescript-ar plugin provides. Check out the plugin’s API documentation for more information on available methods and what they do. Before we wrap up let’s look at one more method.
Adding 3D Models
Adding boxes, text, and spheres to reality is fun, but oftentimes AR apps need to work with custom 3D models. For that, the nativescript-ar plugin provides a custom addModel() method.
WARNING: Finding 3D models and getting them to work with ARKit can be painful. The plugin’s documentation has some suggestions on how to find models and get them working, but it can take some trial and error to get things up and running. The good news is you’re using ARKit, so any existing guides you find about creating models for ARKit will work with NativeScript.
For this demo, first download the three .bae files from the main nativescript-ar demo, and then place them in your app’s app/App_Resources folder as follows.
,
,
Next, open your home.component.ts file back up and replace its content with the following code, which calls the nativescript-ar plugin’s addModel() method.
,
,
TIP: Replace Car.dae with Bal.dae or Tree.dae to experiment with other models.
When your app refreshes on your device you’ll have the ability to place a virtual car into your local reality. Here’s what it looks like if I use the app to put a new car in my driveway.
,
,
Wrapping Up
There’s a whole lot you can do with AR in NativeScript. In this article, you learned the basics, such as how to detect planes, how to add boxes, and how to work with simple 3D models.
Have you done anything cool with AR in NativeScript? If so let us know in the comments. And stay tuned for some more advanced coverage of AR on this blog in the future!
Source: DZONE