How to manipulate a virtual element during an augmented reality experience

Introduction

In this article I will briefly show you how to move an object in augmented reality directly with the touch of the device.

The project will be developed with Unity for Android devices with the ARCore and AR Foundation libraries.

Development

To manipulate objects we should in principle create an AR project where we can AR Object Control Tool, available on Github, makes it easy to scale and rotate objects in AR.

Prerequisites

Before starting development is necessary

import the ARCore and AR Foundation libraries into the project,
set the build for android devices,
add to plugin provideres ARCore,
remove Vulkan from the Graphics API and set the build only for devices with Android starting from version 8,
and finally you may need to set up a build for arm64 devices (scriptingBackend IL2CPP is required).

Finally, you need to import the package available at the following link into your project: https://github.com/LeoSery/AR-Object-Control-Tool--Unity3DTool/releases/tag/V3.0

Scene setup

As with all AR projects, start by removing the Main Camera.
Add XR->AR Session Origin and XR->AR Session to the scene. From the GameObject > XR > menu add AR Session and AR Session Origin to the scene.
To enable plane detection add the AR Plane Manager script to the AR Session Origin object. I also recommend changing the DetectionMode field to Horizontal.
To interact with trackable functionality add the AR Raycast Manager script to the AR Session Origin object.
To display the position of the planes in our AR scene we need a Plane Prefab so that the AR Plane Manager script can display the position of the plane in our AR scene; so let’s add an XR > AR Default Plane into the scene. Drag the newly created object into the project asset to create a prefab with the AR Default Plane and delete it from the Scene Hierarchy.
Add the newly created AR Default Plane to the AR Plane Manager script, as shown in the figure below.

Plane Manager

Now we need an object to appear and then manipulate on the AR environment. To do this we create a cube and call SpawnableObject.

Remember to reduce the scale to 0.1 for each axis to have a good view when aiming the object in AR.

In my example instead of the cube I will use the ReadyPlayerMe avatar, if you want to know how to use the avatar instead of the cube look at this article: https://www.francescogarofalo.it/post/come-creare-un-avatar-personalizzato -and-use-it-in-unity-with-readyplayerme/

Create a Spawnable tag and add it to SpawnableObject.
To make the Spwnable Object appear you need to create a script that takes care of this Spawnable Manager.cs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;
public class SpawnableManager : MonoBehaviour
{
    [SerializeField] 
    ARRaycastManager m_RaycastManager;
    List<ARRaycastHit> m_Hits = new List<ARRaycastHit>();
    [SerializeField] 
    GameObject spawnablePrefab;
    Camera arCam; 
    GameObject spawnedObject;

    void Start()
    {
        spawnedObject = null; 
        arCam = GameObject.Find("AR Camera").GetComponent<Camera>();
    }

    // Update is called once per frame
    void Update()
    {
        if (Input.touchCount == 0)
            return;
        RaycastHit hit;
        Ray ray = arCam.ScreenPointToRay(Input.GetTouch(0).position);

        if (m_RaycastManager.Raycast(Input.GetTouch(0).position, m_Hits))
        {
            if (Input.GetTouch(0).phase == TouchPhase.Began && spawnedObject == null)
            {
                if (Physics.Raycast(ray, out hit))
                {
                    if (hit.collider.gameObject.tag == "Spawnable")
                    {
                        spawnedObject = hit.collider.gameObject;
                    }
                    else
                    {
                        SpawnPrefab(m_Hits[0].pose.position);
                    }
                }

            }
            else if (Input.GetTouch(0).phase == TouchPhase.Moved && spawnedObject != null)
            {
                spawnedObject.transform.position = m_Hits[0].pose.position;
            }
            if (Input.GetTouch(0).phase == TouchPhase.Ended)
            {
                spawnedObject = null;
            }
        }
    }

    private void SpawnPrefab(Vector3 spawnPosition)
    {
        spawnedObject = Instantiate(spawnablePrefab, spawnPosition, Quaternion.identity);
    }

}

Create an empty game object in your scene and name it SpawnManager.
Insert the previously created Spawn Manager script into the SpawnManger object.
Add in the script field Spawn Manager.cs Raycast Manager: AR Session Origin and in the field Spawnable Prefab:Spawnable Object.

Spawn Manager

Now you can test that the SpawnableObject is correctly instantiated.

Manipulate the SpawnableObject

With the AR Object Control Tool package you can easily modify some characteristics of the SpawnableObject, created in the previous steps of this article, in runtime.

Setup

To enable the activities of

Scaling
Rotating
Replacement
Movement

you can add the Movement Manager.cs script to the Spawnable Object.

By enabling the Scaling, Rotating and Replacing options you can easily manipulate the object in AR.

Movement Manager Script

Conclusion

By adding the MovementManger script to the object we want to manipulate we can easily modify an object in augmented reality at runtime by changing its position, size and rotating it to our liking.

This way we can easily create any type of augmented reality game to try in uncontrolled environments.

Fonti

Thanks :)