Nral for the partially sighted RGB Camera zoom live view pass-through?

General
1

Hi there all. First off, I would have gotten this nreal light with a Vodafon contract here in Germany but then realized I would not be able to experiment with it using unity as I would need also a computing unit. Very sad that consumers cannot experiment with the glasses in Unity.
So now to why and my questions.

I am partially sighted (Stargardt’s). So I was wondering if using Unity could I grab the RGB camera feed from the Nreal and have it live in my view? Then I could zoom that video to enlarge my surroundings. An example for this is while waiting for a bus I could zoom into the oncoming bus a be able to read if it is the right bus or not. Also reading signs and generally enlarging stuff from a distance.

I did this a while back with a GearVR and Unity using an asset called NatCam and this worked perfectly but sadly I am not going to strap a big chunky GearVR on my head while waiting for a bus. But the Nreal does appeal to me as being small and light enough to use.

The question however is that the camera is only 5MP, so I do not know if that would be enough.
And also the FOV is only 52 deg and not like 90 with the GearVR But maybe 52 is enough if I can pull the image closer in the glasses.

  1. How big would you describe the screen in the Nreal? I mean, how close can you get the image in front of you? Is the image like 3 meters away, closer or further? What size screen would you estimate it to be, like an 80inch or more at what distance?

  2. Can you access the camera in Unity just like you would an android camera on a smart phone. Maybe someone has Unity NatCam and can try this out.

  3. Will it ever be possible to use Unity and Nreal without the Computing Unit?

  4. I live in Heidelberg Germany, so maybe someone here lives near me and could let me take a look through the glasses if my idea would be possible or not.

  5. Any technical people out there that think it could be possible to swap out the 5MP rgb with a better 16MP or bigger camera? Most probably not possible but I thought I would ask anyway.

So with the first consumer version of such glasses I and many other millions of people are getting closer to our dream of a pair of sunglasses that can enlarge and zoom into our surroundings. This would be a massive game changer for all of us nearly blind people. And that is an understatement!
We are probably going to have to wait a while still for the glasses to be AIO self-contained with a better camera and at least 90 Fov or more.

But It’s a start in the right direction.
People can fly into space nowadays but we partially sighted cannot get a pair of digital glasses.
That’s life……….

Thanks for reading and any suggestions anyone may have.

2

Hi,
the Nreal glasses never work standalone, you always have to connect it to a phone or a development kit. At least for the “Nreal Light”, I am not sure about the newly announced “Nreal Air”, but I guess it’s the same there.
But if you have a phone that supports the Nreal Light, then you can use Unity for development and test it with your phone+glasses.

You can access the RGB Camera of the Nreal Light, but I think it is not 5MP, the resolution is 1280x720. But I could be wrong here.

When you look through the glasses, you will see the world around you (because they are see-through glasses). Additionally you can blend in virtual objects. One object could be a virtual screen in your case. Since it’s a virtual object you can place it at any distance and size you want. You are only limited by the field of view 52°. Using some trigonometry you can calculate the maximum screen size at a given distance (width = 2distancetan(52°/2) )

For your usecase, having an image overlay in your field of view, I am not sure if it will look good in real life usage. You will always see the real world background behind the virtual screen. But it’s worth a try I guess.

3

Thanks for your reply.
Can someone tell me then, If I was watching a film what would be the feeling of the maximum size and distance of the screen. Like watching in a cinema.

An estimate would help me like is it an 80” screen at 2 meters or can we have more?

4

If the formula I posted is correct, then you would get an equivalent of an 88" screen at 2m distance. (Width = 1.95m)
I don’t have a TV that large to confirm it :smiley:

5

Thank you, that sounds pretty good for a start and could be just enough to try out a few ideas. I then also look forward to Nreal boosting up the FOV in newer versions but it also looks like that even though the Nreal Air will have more FOV, they get rid of the cameras. VERY SAD Indeed. Rather stick a 64MP camera on that please. I am greedy, so also with optical zoom and optical stabilizers please.