depth image and color image

[fwang22]

Hi，

I can obtain the color image and depth from intel realsense viewer. but there are three questions about it：

first, the .bag file contains color images and depth images,but their numbers don't match.why?

second,I want to convert the depth image to pointcloud, is there a program?

thrid, In the viewer, the frame rate of the camera is only 30. This cannot be changed in the viewer. How to deal with it?

thank you ,

Best Regards,

Nevyn

[MartyG · Accepted solution]

1. It sounds as though the color and depth needs to be aligned. Intel support team member Alexandra offers advice in the link below about how to do this in the ReealSense Viewer using the 3D point cloud mode.

2. As mentioned above, you can switch the depth to a point cloud in the RealSense Viewer by switching from 2D to 3D mode. You can then export the point cloud as a .ply format file, similar to how you can save a PNG image in the 2D mode.

3. If you are using the depth resolution of 1280x720, this only supports a maximum of 30 FPS. To gain access to 60 FPS, you should drop to the next resolution down, 848X480

[fwang22]

Hi, MartyG,

Because I have obtained the .bag files, I want to convert the depth image of it to point cloud ,and I think it need some programs.

Thank you!

[MartyG]

The SDK has a tool called Convert for converting a bag file to other formats, including .ply.

https://github.com/IntelRealSense/librealsense/tree/master/tools/convert librealsense/tools/convert at master · IntelRealSense/librealsense · GitHub

[fwang22]

Hi MartyG,

when I run the re-convert.exe in windows, there will be errors, but they can't be solved.Do you know how to deal with it?

[MartyG]

In this kind of application, you need to put letters after the filename to tell the program what you want it to do when it runs. For example, to convert the bag to a ply, put -l on the end (a lower-case L) and the path that you want to save the file to (the path in the example below is your desktop):

You could highlight the line below and do Ctrl-C to copy it, and then go to the Convert window and do Ctrl-V to paste the command in.

rs-convert.exe -l C:\Users\wangfengping\Desktop\

[fwang22]

Hi MartyG，

The question has been solved.Thank you very much!

Nevyn

[MartyG]

Awesome, I'm very glad you got a solution.

[fwang22]

Hi MartyG，

Thank you for your help!

But the earlier question: the number of the depth image and the number of the color image are different.and the suggest of you is about pixels align.

so I don't know how to deal with it.

Nevyn

[MartyG]

I went back to the advice from Intel support member Alexandra about aligning in the Viewer that I linked to earlier and worked out the process she was describing. Left-click on the images to view it in full size.

STEP ONE

In 2D mode in the Viewer, activate both the Stereo and RGB streams.

STEP TWO

Go into 3D mode to see the point cloud. Click on the menu beside 'Texture Source' at the top of the window to drop it down. Because you activated the RGB stream in 2D mode, the 'Intel RealSense Color' mode will now be available for selection. Select the Color mode.

The two streams should now be aligned.

[fwang22]

Hi MartyG,

I did that last time, but there are still some inconsistencies but not in the pixels.

For example, there are 66 depth charts and 65 color charts in the.bag file.

Thank you!

Nevyn

[MartyG]

A case over on the RealSense GitHub forum says that depth and IR are perfectly synced by timestamp, but depth and color sync with a 'best match' because they are on separate sensors.

https://github.com/IntelRealSense/librealsense/issues/1548 RGB and Depth frames not synchronized · Issue # 1548 · IntelRealSense/librealsense · GitHub

[fwang22]

Because I obtain the .bag files by record in the intel realsense viewer, how to choose the frames have the smallest temporal difference?

[MartyG]

You could try using a post-processing filter in the Viewer.

[fwang22]

Hi MartyG,

All the problems I encountered have been solved. Thank you very much for your timely reply.

Best Regards,

Nevyn

[MartyG]

Great! Glad I could help. Feel free to return to this forum any time that you need to if you have further questions

[fwang22]

Hi MartyG,

I just tested a few .bag files ,the numbers of them are all the same,but there are still some inconsistencies.

[MartyG]

Are the inconsistencies large enough to visibly affect the results when the bag is played back, or are they just annoying because you know they are there even if they cannot be seen.

[fwang22]

Now I just want to get the same numbers of depth image and color image by the convert program

https://github.com/IntelRealSense/librealsense/tree/master/tools/convert librealsense/tools/convert at master · IntelRealSense/librealsense · GitHub ,

but unfortunately, the fact of the matter is that the number of depth image and color image is sometimes too large after conversion.

And when I want to the convert program to convert the .bag file to color , there are always failed.ans shows"output not defined",but the form is ' rs-convert -i *.bag -c ', and it needn't path.

[MartyG]

I wonder if we are over-complicating this. Instead of recording a bag in the RealSense Viewer and converting it to a ply, it would make more sense to export a ply point cloud file directly from the 3D mode of the Viewer once color and depth have been aligned and temporal post-processing applied.

If you need a bag, another approach would be to record the bag in 2D mode and then load the bag back into the Viewer and export a ply from 3D mode after setting the alignment and temporal filter settings (since a bag is a recording of live camera data, so making a ply from a bag should provide the same results as making it from live camera data).

[hermione]

Hi MartyG,

My doubt is that each ply file exported from the Intel RealSense Viewer corresponds to a specific moment in time when the camera captures the depth and color information right?

But how to get the point cloud data for all the time stamps all together?