With all the hype around augmented reality (AR) and more recently mixed reality (MR), I decided to take the plunge into Magic Leap development. Thought to share my initial experiences.
Update: Wrote this in Lumin SDK 0.14.0 but just upgraded to 0.15.0. While there were some breaking changes, the performance of the simulator is night and day better (pretty much usable now).
Magic Leap Creator
The first step is to sign up to become a Magic Leap Creator which gains you access to their developer portal (Creator Portal). Once you login, you will have access to their documentation (Learn), forum (Forum), and downloads (Downloads).
Browsing through the materials, it is clear that they are just building out their developer portal; documentation is fairly shallow and their forum is not terribly active.
Download and Install Platform
There are currently three development platforms:
- LuminSDK: Develop using a low-level C API
- Unity: Develop using Unity (and C#)
- Unreal: Develop using Unreal Engine (and C++)
Having just learned Unity through the series Learn Unity 3D for Absolute Beginners, I chose the Unity option. The relevant software components, per documentation, are (bold ones are used directly during development):
- Magic Leap PackageManager: Used to install (and upgrade) Magic Leap software
- Lumin SDK: Installed from the Magic Leap PackageManager; includes a number of Magic Leap utilities
- Magic Leap Remote: Included in the Lumin SDK, includes the Simulator (simulates the ML1 hardware)
- Magic Leap Unity Package: Installed from the Magic Leap PackageManager; library integrating Unity and Magic Leap
- Unity API Documentation: Installed from the Magic Leap PackageManager; API reference of Magic Leap classes in Unity
- Unity Editor (Unity 2018.1.Ob12-MLTP5): Installed from Unity’s web site; a special version of Unity supporting the Magic Leap. Used to build Unity projects.
- Visual Studio for Mac: Installed through the Unity installer. Used to program functionality into Unity projects (using C#)
Once all the software is downloaded and installed we can begin development. The first step is to start the Simulator (feature of Magic Leap Remote); specifically:
- Start Magic Leap Remote
- Press Start Simulator button (interestingly enough, with the Simulator started my laptop’s fan kicked into high gear)
- Because developing with Magic Leap involves interacting with the physical world (spacial computing), we will want load a sample room into our Simulator using the left-most button in the Mini Map panel’s button bar (their online documentation is out of date).
- Finding the file location to the sample rooms took a bit of effort (ended up using Magic Leap PackageManager to open the Lumin SDK folder installation folder. From there you navigate to: VirtualDevice\data\VirtualRooms\ExampleRooms
- Now our simulator is ready:
Assuming you have some basic familiarity with Unity, following their Magic Leap Remote — Hello, Cube! Unity® Edition documentation is straightforward; amounts to creating a cube, changing the build environment, and playing the scene in Unity.
Coordinates and Scale
When it comes to scale, game engines and their editors are designed with the key assumption that the user will be working in real world scale, (or at least very close to it.) Systems such as physics, lighting, and particles can quickly break down and yield unexpected results if the working scale is too extreme. The same goes for navigating your world or level within the editor! Panning, tracking, and dollying through the viewport can either be extremely tedious or far too sensitive to work with at very large or very small scales.
— Magic Leap — Working with Scale
The first thing to observe is that 1 game unit equals 1 meter; notice that our cube looks about 1m x 1m x 1m in the room.
Next is that y coordinate is 0 units at the floor; thus setting the cube’s y coordinate at 2 units puts the cube at eye level.
At this point, it is not clear where the x and z coordinates are set to be 0 units (maybe the middle of the room); but in the simulator you are initially positioned at x and z at 0 units. So with the cube’s x coordinate equal to 0 units and z coordinate equal to 3 units, the cube appears 3 meters in front of you.
Next Learning Steps
The first thing one needs to learn is the basics of Unity itself; while not perfect the no-cost Learn Unity 3D for Absolute Beginners was sufficient for me.
Then, the Magic Leap documentation introduces you to specific Magic Leap APIs through a series of examples.
- Control: Demonstrates receiving input from the Control and sending haptic feedback to it.
- Gesture Input: Demonstrates preferred methods for using discrete hand gestures.
- Image Tracking: Demonstrates how to use image recognition and tracking images.
- Meshing: Demonstrates preferred methods for using the meshing functionality.
- Planes: Demonstrates how to detect surfaces and display planes.
- Raycast: Demonstrates preferred raycasting methods.
I am looking forward to digging into these example soon.