how to match the rgb image with the depth image

[锦李]

Hi,I want to match the rgb image with the depth image. Simply say, I want to know the depth value of each pixel on the rgb image, but the rgb image and the depth image are not point-to-point correspondence, so you can tell me how to use the SDK development? I did not find the related code in sdk. thanks

[MartyG]

The SDK has a sample program called Align that aligns the depth frame with the color frame.

https://github.com/IntelRealSense/librealsense/tree/master/examples/align librealsense/examples/align at master · IntelRealSense/librealsense · GitHub

[锦李]

thanks, but I want to get the source code,I want to know the Conversion matrix between rgb image and depth image.Could you tell me ?

[jb455]

The code where the alignment happens is here: https://github.com/IntelRealSense/librealsense/blob/master/src/proc/align.cpp# L31 librealsense/align.cpp at master · IntelRealSense/librealsense · GitHub

There's no 'conversion matrix' - the algorithm uses the properties of each camera (intrinsics) and the physical relationship between them (extrinsics) to map each point individually.

[锦李]

THANKS

[jb455 · Accepted solution]

Check out the align example: https://github.com/IntelRealSense/librealsense/tree/master/examples/align librealsense/examples/align at master · IntelRealSense/librealsense · GitHub

This document is also useful to understand the mechanics of it: https://github.com/IntelRealSense/librealsense/wiki/Projection-in-RealSense-SDK-2.0 Projection in RealSense SDK 2.0 · IntelRealSense/librealsense Wiki · GitHub

[JYu33]

Hello, I would like to ask how to save the depth image of realsense in the format of TXT file. Each value in the TXT file represents the distance from this point to the depth camera.

[idata]

Hi YuJionghua,

Once you have the depth values, how you write them to a TXT file depends on what programming language you are using. There is no RealSense-specific example for writing depth values to a text file. The major steps to do this, in any language are:

1. stream depth from camera
2. find depth values
3. open or create text file
4. output depth values to text file
5. close text file

Hope this helps.

Regards,

Alexandra

[JYu33]

Hi Alexandra,

Thank you for your reply, I'm using LabVIEW now, and I want to use it to save the depth image of realsense in the format of TXT file. Each value in the TXT file represents the distance from this point to the depth camera.

[MartyG]

Was the YouTube guide I showed you last week any help, YuJionghua?

[JYu33]

Emmmm, it doesn't seem to work. I just don't know how to export the depth map data in labview right now. Could you give me some suggestions？

[MartyG]

I located the sections of the LabVIEW 2018 manual that refer to text files.

General principles of text files

http://zone.ni.com/reference/en-XX/help/371361R-01/lvconcepts/fileio_text_files/ Text Files - LabVIEW 2018 Help - National Instruments

Creating a text file

http://zone.ni.com/reference/en-XX/help/371361R-01/lvconcepts/creating_text_files/ Creating Text Files - LabVIEW 2018 Help - National Instruments

Reading from text files

http://zone.ni.com/reference/en-XX/help/371361R-01/lvhowto/reading_from_text_files/ Reading from Text Files - LabVIEW 2018 Help - National Instruments

Writing to text files

http://zone.ni.com/reference/en-XX/help/371361R-01/lvhowto/writing_to_text_files/ Writing to Text Files - LabVIEW 2018 Help - National Instruments

[JYu33]

Thank you, I'll try it.

[JYu33]

Hi MartyG, I have just get the depth image, but it doesn't look very good, I want to use this to extract the outline of the object, but I can't do this now. Can you give me some advice?