Is the depth aligned to RGB on ZR300?

[HWade]

Hi, is the depth data coming in the ros topic camera/depth/image_raw already aligned to RGB?

I am experiencing large displacements but this might also come from the bad extrinsics/intrinsics of the default camera calibration.

Thanks alot!

[idata]

Hi hanneswader,

Thanks for contacting the Intel RealSense Community, allow me to investigate this inquiry on my end and I'll come back as soon as I get more information on this issue with the ZR300 camera. We hope you have a great day! Regards, Leonardo L

[idata · Accepted solution]

Hi hanneswader,

Thanks for your time, we have analyzed this case and we have the following information for you in regards to the depth data aligned to RGB: The depth data from the ros topic camera/depth/image_raw is not aligned to RGB. To align the depth and color images, users must use the Projection API in the Intel RealSense SDK for Linux (https://software.intel.com/sites/products/realsense/sdk/developer_guide.html# SDK_Spatial_Correlation_and_Projection https://software.intel.com/sites/products/realsense/sdk/developer_guide.html# SDK_Spatial_Correlation_and_Projection) as demonstrated in the Projection sample: https://github.com/IntelRealSense/realsense_sdk/tree/master/samples/src/projection_sample https://github.com/IntelRealSense/realsense_sdk/tree/master/samples/src/projection_sample. The Projection Tool is a GUI that shows how projection works: https://github.com/IntelRealSense/realsense_sdk/tree/master/sdk/src/tools/projection_tool https://github.com/IntelRealSense/realsense_sdk/tree/master/sdk/src/tools/projection_tool.Feel free to test this API samples and let us know if you need more assistance with this. We hope you find this information helpful. Regards, Leonardo L

[idata]

Hi hanneswader,

We are writing to follow up on your thread, let us know if the information provided worked for you or if you need further assistance with this. We'll be looking forward to your reply. Have a great day! Regards, Leonardo L

[HWade]

Hi Leonardo,

thank you very much for your answer!

Sorry for my delay, I found very little time this week for this.

However, the examples are great!

In the projection sample, I see that each pixel in the depth image is processed to a 3d point:

https://github.com/IntelRealSense/realsense_sdk/blob/master/samples/src/projection_sample/main.cpp# L48 realsense_sdk/main.cpp at master · IntelRealSense/realsense_sdk · GitHub

But there is no actual projection using the camera intrinsics. The code just uses the pixel coordinates as x and y coordinates in 3D world, which I dont quite understand.

I will do some experiments with the projection tool today and report the outcome here.

Thanks again!

Update: Ahh, I just looked into the source of projection_->map_depth_to_color and saw, that it actually expects the image coordinates and the depth value in m.

[idata]

Hi hanneswader,

More than welcome, sure, feel free to test the projection tool once you have time and let us know how it goes after testing. Have a great day! Regards, Leonardo L