Threshold with OpenCV

[LFerr16]

I am working in a project at my university and the main idea is to measure the distance between two objects (supposed to be the only two objects in the environment). As I am not very familiar with the library from RealSense, I tried a solution with OpenCV itself, so I planed to identify the objects with the watershed segmentation algorithm:

https://docs.opencv.org/3.4/d2/dbd/tutorial_distance_transform.html

I obtained satisfactory results, but to improve the segmentation I would need to make a threshold of the image by distance. However, when I convert the data to a Mat image I convert the distance to a Vec3b value of the colorized depth data, but I don't know how to access the depth data (the value in meters) from the Mat to make a threshold or to apply a threshold filter before the convertion.

rs2::frameset data = pipe.wait_for_frames(); rs2::frame depth = data.get_depth_frame().apply_filter(color_map);   const int w = depth.as<rs2::video_frame>().get_width(); const int h = depth.as<rs2::video_frame>().get_height();   Mat image(Size(w, h), CV_8UC3, (void*)depth.get_data(), Mat::AUTO_STEP);

[Eliza_D_Intel · Accepted solution]

Hello LFerr16, Thank you for your interest in the Intel RealSense Technology. I found this GitHub issue - https://github.com/IntelRealSense/librealsense/issues/2634#issuecomment-434267985 that "in terms of depth sensing solutions, we are not yet in a place where you could just take the raw depth map and use it as a threshold. Main issue is that most depth cameras available today are composed from several sensors, each at slightly different location. This means that you will always going to get pixels that one sensor sees but the second does not, resulting in black pixels." You can also check this thread for another approach on this subject: https://github.com/IntelRealSense/librealsense/issues/2338 I apologize we cannot be more help but we are technical support not computer vision experts. We can help with camera functionality issues and basic programming using the SDK 2.0 but your question is about a custom advanced algorithm. I recommend to ask your question on other forums for OpenCV or StackOverflow Thank you, Eliza