Head Up Display with NReal and Unity

Development & Help
1

Has anyone got a Head Up Display (HUD) working in AR using Nreal glasses and Unity?

We’ve tried attaching a Canvas + TextMesh components to the NRCameraRig but this is not showing the overlayed HUD when running on the Nreal glasses.

The HUD does appear when running via the Unity simulator.

2

I’ve done something similar (though I don’t lock my menu to the headset but have it smoothly follow the headset around with a delay). You probably need to make the Canvas a World-Space Canvas and then scale the UI down to world-scale, so the UI elements appear in the world with you rather than being rendered as a camera overlay.

3

Thanks Josh but we need to have a 2D HUD, fixed at all times to the viewer’s display that is not part of the 3D AR world space. We know it’s possible, since Nreal are doing this to display their home screen menu. Any other ideas of how we could achieve this?

4

I did not manage to achieve overlay behavior from standard Unity UI in my apps, but I did what josh suggested as a workaround.

Just create your UI in world space, and add a script on the main canvas to strictly follow the camera movement. There is even a beneficial side effect: you can setup the depth of your HUD to limit eye strain.

The down side: your HUD might clip the environment (solvable with some clever rendering order/shader shenanigans I suppose)

Here’s what I use in my app. Drop this component on your main HUD canvas, set up in world space, and it should stay in place (didn’t touch it in a few month, not sure whether the “positionOffset” membre is useful or not):

using UnityEngine;
using NRKernal;
public class StrictCameraFollower : MonoBehaviour
{
    private Transform cameraCenter { get { return NRInput.CameraCenter; } }

    private Vector3 positionOffset;
    private void Awake()
    {
        positionOffset = Vector3.zero;
    }

    void Update()
    {
        transform.position = cameraCenter.position + positionOffset;
        transform.rotation = cameraCenter.rotation;
    }
}

Edit: And don’t forget to set up the canvas Z position to manage depth

5

Thanks Julien, this is working now :slight_smile:

6

I’m using Canvas + Canvas Raycast Target + Move With Camera