ask about the origin of pointcloud

[NMan]

Hi all,

I'm a new D435 user and I want to ask a about the coordinates of pointcloud.

What is the origin of the pointcloud returned by the "calculate" function?

Many thanks in advanced!

I use this code for getting point cloud data

rs2::pointcloud pc;

rs2::points points;

rs2::pipeline pipe;

pipe.start();

auto frames = pipe.wait_for_frames();

auto depth = frames.get_depth_frame();

points = pc.calculate(depth);

[MartyG]

As the point cloud uses the depth and color frames to calculate the cloud, I would assume that it uses the camera's origin point.

The data sheet document for the 400 Series cameras says "The left and right imagers capture the scene and sends imager data to the depth imaging processor, which calculates depth values for each pixel in the image by correlating points on the left image to the right image and via shift between a point on the Left image and the Right image."

The calibration guide for the 400 Series, meanwhile, validates this by saying: "Assume left camera is the reference camera and is located at world origin."

The baseline is the distance between the center of the left and right imagers, which is 50 mm on the D435.

[NMan]

You mean the origin of point cloud is located at the center of the left imager of the camera.

How many millimeters deep into the camera from the glass surface is the origin located?

[MartyG]

The origin should be the baseline (the center between the imagers). As far as I am aware, the mm distance that the sensor is behind the glass isn't a factor in determining the origin. The camera probably takes account of that thickness and treats the outside of the glass as the '0' distance origin..

[NMan]

Thanks for your answer.

The camera 3D coordinate system origin is located at the center between left imager and right imager

So how can I find (x=0,y=0,z=0) point in the camera?

Sorry, my English is bad.

[MartyG]

Your English is fine. )

The origin point between the imagers is 0, 0, 0. The Projection guide that describes how the RealSense SDK 2.0 calculates coordinates says:

"Within this space, the positive x-axis points to the right, the positive y-axis points down, and the positive z-axis points forward. Coordinates within this space are referred to as "points", and are used to describe locations within 3D space that might be visible within a particular image."

https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0 Projection in RealSense SDK 2.0 · IntelRealSense/librealsense Wiki · GitHub

I think that if you wanted to get the real world 3D coordinates of an image that the camera was seeing, you would use the instruction rs2_deproject_pixel_to_point (which is described in the page I linked to above). I will ask our stream programming expert to be sure - is that correct, jb455

[PSnip]

Hi Nosman,

When you have multiple cameras in a system, each camera will have its own center origin. Since Left Imager, Right Image, and Color camera are on a straight line [which Marty referred as baseline], the camera centers for 3 cameras are located at same point in y & z axis. Only displacement is on x-axis.

Refer last section on following link for a detailed explanation.

https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0

If you want to understand the x-displacement between any two cameras [Left Infra, Right Infra & Color], you can refer to the extinisics of these cameras. Additionally the Depth Camera is assumed to be aligned to the Infra Imager close to RGB sensor. It is just a convention, as physically there is no separate depth camera, depth is only estimated based on Infra1 & Infra2.

e.g. on my D435, I get following extrinsics for translation [rotation is Identity Matrix],

1. depth_to_infra1:

[0.0, 0.0, 0.0]

2. depth_to_infra2:

[-0.0500330813229084, 0.0, 0.0]

3. depth_to_color:

[0.014696473255753517, -0.00015400485426653177, -0.0001506415574112907]

Above implies that

1. Depth camera center of origin is assumed to be aligned to Infra1 sensor [the Imager next to color camera]

2. Infra2 sensor is 5 cm left of Infra1

3. Color sensor is about 1.47 cm right of Infra 1

I hope above adds some more clarity to what was already being discussed between you and Marty?

[NMan]

Thank you so much for your answer!

I want to find the position of origin of real world 3D coordinates of point cloud data that calculated from depth data.

Like your D435, the origin is on the baseline and located at the center of left imager (next to RGB sensor), but how can I find how depth from the glass surface is the origin located?

And can you share me the code for get extrinsics (depth_to_infra1:, depth_to_color) ? I'm a new user of D435 so I don't farmiliar to API of this camera.

[PSnip]

Hi Nosman,

Did you fully read the wiki page which I shared earlier?

https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0

I suggest that you read the above along with realsense SDK API documentation. This will answer all your questions.

To elaborate further, if you want to know the 3-D co-ordinates corresponding to a particular pixel in Depth Image, you could call the function rs2_deproject_pixel_to_point()

https://github.com/IntelRealSense/librealsense/blob/5e73f7bb906a3cbec8ae43e888f182cc56c18692/include/librealsense2/rsutil.h# L46

[Above is from the wiki page shared earlier]

Also, to get extrinsics, you could use get_extrinsics() call, details here

https://github.com/IntelRealSense/librealsense/blob/5e73f7bb906a3cbec8ae43e888f182cc56c18692/examples/sensor-control/api_how_to.h# L209

[Above information again based on the wiki page]