Unity ARKit By Example: Part 4

Exploring image detection and tracking (and object tracking).

Image for post
Image for post

This article is part of a series starting with Unity ARKit By Example: Part 1.

Image Tracking

In this example we will use the imaging tracking API from Unity-ARKit-Plugin; we will use its ability to detect and track images in the physical world. Tracking was added in ARKit 2.0 and released with iOS 12.

In particular, when the target image is in the camera viewfinder, the application will place and keep a 30 cm cube over the image.

Image for post
Image for post

Contrary to the previous examples, we will keep all the assets (there a number of them) for the application in a separate folder; Assets > Examples > ImageTracking. We start by saving the Hello World scene as ImageAnchor into the application folder.

We then remove the Cube, Canvas, and EventSystem GameObjects from the scene.

We drag the the target image, e.g., jpg, into the application folder.

Image for post
Image for post

note: On many of my sample projects I use cat images; I just find them fascinating animals.

We use the Unity menu, Assets > Create > UnityARKitPlugin > ARReferenceImage, to create an image reference; saving in the application folder, e.g., with name cat. The reference image, in addition to defining the Image Texture (the target image of the cat), it defines the Physical Size, e.g., 0.3 is 30 cm, in the physical world.

We use the Unity menu, Assets > Create > UnityARKitPlugin > ARReferenceImagesSet, to create an a collection of reference images; saving in the application folder, e.g., with name ImageAnchor (for lack of a better name). In this case, it has a Size of 1 and with Element 0 referring to the image reference we created.

To enable the application to track the images (actually one of them) defined in the images reference set, we add it to the ARCameraManager (GameObject) > Unity AR Camera Manager (component) > Image Tracking > Detection Images (property). We also ensure the property, Maximum Number of Tracked Images, is set to 1.

Image for post
Image for post

As with the previous article, we need to create the 30 cm Cube prefab in the application folder.

Finally, we need to create an empty GameObject, e.g., ImageAnchor, and a script, ImageAnchor, in the application folder. We add the new script as a component to the new GameObject.

Observations:

In order for the script to operate, we need to assign the image reference (cat) to the public referenceImage property. Likewise we need to assign the Cube prefab to the public anchorPrefab property.

Image for post
Image for post

The complete set of objects in the application’s folder are:

Image for post
Image for post

note: In hindsight, it might have been more sensible to use different names for the different types of objects.

We now follow the instructions in the section The Build Process of the article Unity ARKit By Example: Part 1 to build and load the application to an iOS device. A couple of things to keep in mind:

Now running the application on an iOS device, we can see the application working.

note: We have to build and load the application to an iOS device to test the application as ARKitRemote does not support image detection and tracking.

Object Detection

Turns out that object detection is very similar to image tracking; the primary difference is that object detection makes use of a point cloud as opposed to an image as the target.

The following video does a great job of explaining the general workflow (the additional complexity is generating the point cloud target); and given the image tracking example above one can mostly build the equivalent object detection example.

For example, I followed the instructions to create a point cloud of a dumbbell and then made the application create a cube when it detects it.

Image for post
Image for post

The following are the assets we end up with:

Image for post
Image for post

Observations:

Wrap Up

Looking through the documents and tutorials, there are a number of features that we did not explore:

Figure you can explore these on your own.

Written by

Broad infrastructure, development, and soft-skill background

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store