Plane detection not working after file system access or even simple scene change

Development & Help
, ,
1

Hello everyone !

The Context

I’m working on a point cloud video player, and I have working apps for multiple Android devices

I’m trying to develop a version for the Nreal glasses, and I basically want to place my 3D cloud on a detected plane, just like the HelloMR demo with the Mascot model. In fact, I based my whole scene on the HelloMR demo.

I basically have big files stored in StreamingAssets (the videos), and I have a script copying thoses files from the StreamingAssetPath to the PersistentDataPath when launching the app (there is checks to not copy already existing files, etc…), in order to access them at runtime, as one should when using StreamingAssets in Android. Everything is in the same scene.

Plane detection and file system issue

The plane detection don’t seem to start when I try to access the data (either after the copy, or after only accessing PersistentDataPath) even if I do have storage permission.

Note that I did manage to make the whole app work (plane detection and cloud video player) by hard coding one of the video path in my scene (so it kinda prove the core feature is not the root of the problem). But I can’t implement any kind of video selection without having the whole detection stop.

When I say “stop”, detected plane stay in place, but no more planes are detected. I sometimes get the popup about having not enough anchor point and having to look around, mostly when I get very close to a surface, which make me think that at least part of the detection is still running at that point.

Plane detection and scene loading issue

I tried doing all the file system stuff in a separate scene, without any success, and even found that the simple fact of loading into the HelloMR scene was enough to “break” the plane detection.

Here is an easily reproducible example:

  • I’m using the HelloMR demo scene straight from the SDK. If I compile it alone, or with the other demo scenes (not including hand tracking), I got no issue, everything is working properly.

  • If I add a simple loading scene before HelloMR, I can’t seem to get any plane detection whatsoever.

  • Here is my very basic loading scene setup: NReal Camera Rig, Nreal Input, a floating text in a world space canvas, and a script loading HelloMR after 2 seconds. I get no plane detection whether I use LoadScene or LoadSceneAsync.

The script:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.SceneManagement;

public class WaitNLoad : MonoBehaviour
{
    public float waitTime; //Set to 2 in the inspector
    private float timer;

    void Start()
    {
        timer = 0.0f;
    }

    void Update()
    {
        if (timer > waitTime) SceneManager.LoadScene(1); //1 -> build index for HelloMR
        timer += Time.deltaTime;
    }
}

My Configuration

I’m using the Nreal Light dev kit, and I’m up to date according to the Nreal Launcher app on the dev kit.

Nreal System Version: SDM845-202106292240-837
Nreal Light Controller Version: Dec 25 2019 22:25:28 release
Glasses Firmware Version : 05.1.07.007_20210510
NSDK 1.6.0
Unity 2020.3.15f2 (LTS)

Final Words

Please note I can’t share too much code online, as this is company code, but I can provide a few scripts if needed.

I couldn’t find anything in depth regarding the plane detection, has anyone faced this kind of problems, or have a brilliant idea I could try? Does this bug happen on your end too?

Thank you for your time, and have a great day!

1 Like
2

Plane detection also is not working well over here. Don’t know if it’s the same issue, but I see only horizontal planes and they go far beyond the physical planes.

Some dodgy code in there with coroutine starting with a delay to initialize the triangle list of the plane polygon, I wonder if there’s some timing issue there, doesn’t look too good.

In PolygonPlaneVisualizer.cs

        void Start()
        {
            StartCoroutine(UpdateMesh());
        }

        /// <summary> Updates this object. </summary>
        private IEnumerator UpdateMesh()
        {
            **WaitForSeconds wait = new WaitForSeconds(1f / 5);**
            while (true)
            {
                yield return wait;
#if UNITY_EDITOR
                var center = new Pose(transform.position, transform.rotation);
                GetBoundaryPolygon(transform, m_PosList);
#else
                var center = Trackable.GetCenterPose();
                ((NRTrackablePlane)Trackable).GetBoundaryPolygon(m_PosList);
#endif
                this.DrawFromCenter(center, m_PosList);
#if !UNITY_EDITOR
                if (Trackable.GetTrackingState() == TrackingState.Stopped)
                {
                    Destroy(gameObject);
                }
#endif
            }
        }
3

Hi
I am also facing the same issue were you able to solve this yet?