Get Camera Transformation Matrix

Development & Help
1

Hi,
I am developing a MR application for the Nreal. Here is what I want to do:
I am detecting a real object relative to the camera image that I get from the RGB camera. This works great. But now I want to overlay a virtual object at the right position in Unity. Therefore I need the camera transformation matrix. Can anyone help me with this? Do I have to use the Eye pose or the head pose for this? How can I convert this pose to a 4x4 camera matrix? (extrinsic parameters).
Thanks a lot in advance!
Br, Elisabeth

2 Likes
2

I suggest to use the search function but here:

3

Hi thanks, but I know how to get the intrinsic paramters - I already detected the object relative to the camera right. What I need is to place in in the right position in Unity. Do you know how to do that?

4

I am trying the same but its not accurate yet. Will let you know if I managed it properly but its quite hard. Would you let me know if you managed it, please? :sweat_smile:

5

I am also working with these APIs… I was wondering how you are using the values from GetRGBCameraIntrinsicMatrix to get a projection matrix? The numbers do not look like focal lengths… I ended up calling GetEyeProjectMatrix to get the RGB projection matrix. But everything is a little off.

As for NRFrame.EyePoseFromHead.RGBEyePose - this does seem a little off, it returns:
pos:(0.0, 0.0, 0.0), rot:(0.1, 0.0, 0.0, 1.0) - which looks like a temp value.

There is a function listed in their docs though to convert it to a 4x4 transform:
NRSDK Scripting API — NRSDK 1.6.0 documentation (nrealsdkdoc.readthedocs.io)

Not tried it yet.

6

Sorry for delayed reply. You could check the “NRSDK Scripting API/Device Parameters” in NRSDK document.
The interface of NRFrame.GetEyeProjectMatrix“ could get leftDisplay、rightDisplay、RGBCamera’s PRojectMatrix.

7

Thanks @XREAL-dev - could you share what the values from the call mean:

// rgb camera intrinsic matrix
NativeMat3f rgb_in_mat = NRFrame.GetRGBCameraIntrinsicMatrix();

The returned values are:

1143.524,0,650.8565,
0,1142.766,386.9736,
0,0,1

It looks to me like the 650 and 386 values are cx and cy, however, are the larger numbers fx and fy?

8

Don’t worry @XREAL-dev - I did some reading and found out how to turn these values into a projection matrix:

var intrinsic = NRFrame.GetRGBCameraIntrinsicMatrix();
var fx = intrinsic[0] * SensorWidth / Width;
var fy = intrinsic[4] * SensorHeight / Height;
var cx = ((Width / 2f) - intrinsic[2]) / Width * 2f;
var cy = (intrinsic[5] - (Height / 2f)) / Height * 2f;

I am putting SensorWidth and SensorHeight as 2mm - do you have an actual number for this?

1 Like
9

hey, any updates on this? been a year!