Create a Pokémon GO Style Augmented Reality Game With Vuforia

1. Introduction

In the first post of this series we talked about how awesome Vuforia is for creating Augmented Reality experiences, and now we're ready to practice these concepts in an actual app. In this tutorial, we'll start to play around with Augmented Reality using Vuforia onUnity 3D. We’ll learn how to set up Vuforia and start developing an AR game from scratch, adopting a logic similar to the one used on Pokémon GO!

It won’t be necessary to have any previous experience on Unity or Vuforia to follow this tutorial.

• Mobile App
  Pokémon GO Style Augmented Reality: Markers
  Derek Jensen

1.1. Quick Recap: How Does Vuforia Work?

Vuforia uses the device's camera feed combined with accelerometer and gyroscope data to examine the world. Vuforia uses computer vision to understand what it 'sees' on the camera to create a model of the environment. After processing the data, the system can roughly locate itself in the world, knowing its coordinates: where is up, down, left, right, and so on.

If you don’t know what Vuforia is about, take a look at the first post in this series.

• Mobile Development
  Pokémon GO Style Augmented Reality With Vuforia
  Tin Megali

1.2. What Will We Learn?

This tutorial is divided into two parts. In this one, we'll see some of the particularities of Vuforia on Unity 3D, we'll learn how to set up the environment, and we'll also start developing a small AR game called Shoot the Cubes. We'll pay special attention to the ARCamera Prefab, one of the most important parts of Vuforia in Unity.

In the second part, we'll continue to develop the Shoot the Cubes game, adding interactivity and making it more interesting. This section won't go too much into Vuforia's particularities, as the idea will be to explore some possibilities offered by Unity to create an engaging Augmented Reality experience.

2. Vuforia on Unity

Unity is a popular and powerful game engine that is easy to use and can compile games for multiple platforms. There are some advantages in using Unity to create AR experiences with Vuforia. It's possible to target all Vuforia’s supported systems, including the smart glasses. It's simpler to use, thanks to the Prefabs given by Vuforia’s SDK. Using only Unity is it possible to access all the features available on Vuforia.

2.1. Vuforia Prefabs

You can access all Vuforia's features on Unity using the Vuforia prefabs. All that you have to do is drag the object to the stage and configure it. As the name suggests, prefabs are like templates for creating and cloning Unity objects complete with components and properties. For example, the ImageTarget represents images that can be used as targets. Let's take a look at the Vuforia prefabs available on Unity:

• ARCamera: The most important prefab. It manages the overall AR experience, controlling the render quality, defining the center of the world, the device camera to be used, the maximum targets to be tracked, and so on. In this tutorial we'll concentrate our efforts on understanding how to use this object.
• Targets: All Vuforia targets have their own prefab: ImageTarget, MultiTarget, CylinderTarget, ObjectTarget, UserDefinedTargetBuilder, VuMark, FrameMarker. Those targets will be recognized by the ARCamera and start an action, like exhibiting a 3D object or animation.
• CloudRecognition: Used to access targets defined in the Vuforia cloud system.
• SmartTerrain and Prop: Those objects are used in the Smart Terrain feature.
• TextRecognition and Word: Prefabs used in the Text Recognition feature.
• VirtualButton: Vuforia can understand Targets as buttons that can be physically pressed by the user. This prefab will help you to use this resource.

3. Creating Our First AR Experience

The game that we'll develop is simple, but it illustrates the Augmented Reality principles well, and it will teach us some of Vuforia’s fundamentals. The game’s objective is to find and shoot cubes that are flying around the room. The player will search around for the cubes using his or her device and ‘tap’ to shoot on the boxes. We won’t concern ourselves with score, level or anything like that, but you can easily expand on these aspects of the game yourself.

3.1. Preparing Unity for Vuforia

Before we start playing around, we’ll need to prepare Unity for Vuforia. The process is quite simple, and we basically need to import Vuforia's SDK package and add an ARCamera prefab to our project.

• Create a developer account on Vuforia.
• Make the login and download theVuforia SDK for Unity.
• Open Unity and create a new project called "Shoot the Cubes".
• After the Unity project window opens, go to Assets > Import Package > Custom Package and select the downloaded SDK.
• Import everything.
• Delete the Camera object in the Hierarchy window.
• Go to License Manager on Vuforia’s developer portal and create a new license using your developer account.
• Copy the license key.
• Back to Unity, in the Project window, go to Assets > Vuforia > Prefabs > ARCamera. Select the element and drag it to the Hierarchy window.

• WithARCamera selected, in the Inspector panel, go to Vuforia Behavior (Script), find the field App license key, and paste the license you created in Vuforia's developer portal.

• Click the Apply button near the top of the Inspector pane to add the license key to all ARCamera prefabs on this project.

3.2. Testing if Vuforia Is Working

It's time to check if the environment is working correctly. 

Using Your Computer Camera

If you have a webcam on your computer, you can press Unity's play button to check if the ARCamera is working. It will be possible to recognize targets using the webcam; however, it won't be possible to use any sensor data to test your AR experience. If the camera feed doesn't show the Game window, there is a possibility that your camera isn't compatible with the webcam profile provided by ARCamera. 

Configuring the Application to Run on a Device

The best way to test your Vuforia application is directly on the device. We'll compile the project for Android, but the same steps would apply to iOS devices.

• First, we need to save the Scene that we're working on. Go to File > Save Scene.
• Select the Assets folder and create a new folder called Scenes.
• Save this scene as ShootTheCubesMain.
• Go to File > Build Settings.
• Select Android and click on Switch Platform. If this option is disabled, you'll have to download the desired Unity SDK for the device.

• Click on Player Settings and configure the project in the Inspector window.

• Pay attention to some options: Turn off the Auto Graphics API and make sure that OpenGLES2 is selected for the Graphics API option.
• Type the Bundle Identifier.
• For Android devices, make sure that the Minimum API Level selected is API 9 or greater. You'll also need to use ARMv7 for the Device Filter option.
• If you followed the steps correctly, the project is ready to be built. However, if this is the first time that you're compiling a Unity project for Android or iOS, you have to configure Unity for those devices. Follow this guide for Android and this for iOS.
• To run the project, go back to Build Settings and click on Build and Run.

After the building, the application will be installed on your device. For now, all that you should expect is to see the camera feed on your device without any error. If you've got that, everything worked properly.

3.3. Using the ARCamera Prefab

The objective of the Shoot the Cubes game is to search out and shoot flying cubes using the device's camera and sensors. This approach is similar to the one used on Pokémon GO. To accomplish this, we'll only need to use the Vuforia ARCamera prefab.

There are lots of scripts attached to the ARCamera. For now, the only one that you'll need to understand is the Vuforia Behavior script. Let's take a look at its options:

• App License Key: Where the Vuforia license key should be inserted.
• Camera Device Mode: Controls the render quality of the objects.
• Max Simultaneous Tracked Images: Defines the maximum targets tracked at the same time. Vuforia doesn’t recommend more than five at once.
• Max Simultaneous Tracked Objects: Defines the maximum objects tracked at the same time. Again, Vuforia doesn’t recommend more than five at the same time.
• Load Object Targets on Detection: Loads the object associated with the target as soon as the target is detected.
• Camera Direction: Chose which device camera to use.
• Mirror Video Background: Defines if the camera feed should be mirrored.
• Word Center Mode: The most relevant option for our project. It defines how the system should locate the center of the world. 
• SPECIFIC_TARGET: Uses a specific target as a reference to the world.
• FIRST_TARGET: The first target detected will be used as a reference to the world.
• CAMERA: Uses the camera as a reference point to the world.
• DEVICE_TRACKING: Uses the device’s sensor as a reference to set the world’s positions. This is the option that we need to choose for our little project.

For now, all that you'll need to change in the ARCamera is the Word Center Mode. Click on the ARCamera element in the hierarchy and in the Inspector pane, change the World Center Mode to DEVICE_TRACKING.

3.4. Using the Device's Sensor to Find the Center of the World

Let's add a cube to the stage and test if the ARCamera is working correctly.

• Make sure that ARCamera's position and rotation are set to 0 on the X, Y, and Z axes.

• Create a Cube object from Game Object > 3D Object > Cube.

• Move the cube Position on the Z axis to 10 and 0 on the X and Y.
• Scale the object to 2 on the X, Y, and Z axis.
• Rotate the cube 45 degrees on the X and Y axis.

• You can press the play button to check if the cube is positioned correctly.
• Once you're certain that the cube is positioned correctly, build the project again and test it on the device. To build, go to File > Build and Run.

You'll have to look around by rotating your device to find the cube. You'll notice that the object remains still in the same place, even after you rotate the device away from it. It's as if the cube 'exists' in the real world, but can only be seen with the device camera.

3.5. Setting the Elements' Position According to ARCamera

The first problem with our application so far is that the cube may appear anywhere and the user will have to look around to find it. Since the center of the world is defined according to the device's sensors, we cannot be sure of the actual position of the elements. This is because the user might start off with the device in any orientation, and because the way rotation is measured varies from device to device.

In order to make sure that the AR entities start off in view of the user, the easiest approach is to wait for Vuforia to define the center of the world and to find the ARCamera rotation and then to arrange the starting location of elements according to that orientation.

We'll create a Spawn Manager to define the position of the cubes to be spawned. The manager will define its position according to the ARCamera rotation. It will wait until the rotation is set, and then move 10 units to the front of the camera.

• Create two empty objects with Game Object > Create Empty. Right click on one of the objects you just created and rename it to _SpawnController.
• Change the name of the other empty object to _GameManager.
• In the Project window, select the Assets folder and create a new folder called Scripts.
• In the Scripts folder, create a C# script called SpawnScript.
  
• Drag the SpawnScript to the _SpawnController.
• Double click on SpawnScript to edit it.

First let's add the Vuforia package.

using UnityEngine;
using System.Collections;

// We'll need to use Vuforia package to
// make sure that everything is working
using Vuforia;

public class SpawnScript : MonoBehaviour {

}

To access ARCamera, use Camera.main. Let's create a function to get the camera position and set the cube to be spawned 10 units forward from this point.

public class SpawnScript : MonoBehaviour {

    // Define the position if the object
    // according to ARCamera position
	private bool SetPosition()
	{
		// get the camera position
		Transform cam = Camera.main.transform;

		// set the position 10 units forward from the camera position
		transform.position = cam.forward * 10;
		return true;
	}
}

We'll change the position only once from the Start function. ChangePosition is a coroutine that will wait a small amount of time before setting the position.

    private bool    	mPositionSet;

    void Start () {
		// Defining the Spawning Position
		StartCoroutine( ChangePosition() );
	}

    // We'll use a Coroutine to give a little
    // delay before setting the position
	private IEnumerator ChangePosition() {

		yield return new WaitForSeconds(0.2f);
		// Define the Spawn position only once
		if ( !mPositionSet ){
			// change the position only if Vuforia is active
			if ( VuforiaBehaviour.Instance.enabled )
				SetPosition();
		}
	}

    

Let's test the script:

• Back in Unity, click on the _SpawnController object and use Game Object > 3D Object > Sphere to insert a sphere inside _SpawnController. 
• Select the sphere and make sure its position is set to 0 on the X,Y, and Z axis. 
• Now we'll overlap the cube and _SpawnController so you can notice the importance of the script. Select _SpawnController and set its position to 0 on the X and Y axis and to 10 on the Z axis, the same position as the cube. 

The elements start out overlapping; however, once you build and run the application on a device, you'll see that the _SpawnController and its sphere will appear in front of the camera, and the cube will be in another place. Go ahead and test it! Make sure you're looking at the device right when the app starts.

4. Conclusion

Congratulations, you've created your first Augmented Reality experience. Yes, it's a little rough, but it is working! In this tutorial you've learned how to use Vuforia's main prefab in Unity, the ARCamera. You also learned how to configure it and how to use the device sensors to create the illusion that a virtual object is inserted into the world.

4.1. What's Next?

In the next tutorial we'll improve this principle to create a real game and a more engaging experience. We'll continue to develop the Shoot the Cubes game, adding some interactivity and exploring Unity's possibilities for creating an interesting AR Game. We'll make the cubes spawn and fly around, and we'll let the player search and destroy them by shooting a laser out of the device.

See you soon!

Special thanks for the image vector designed by Freepik, licensed under Creative Commons CC BY-SA.