Problem with the Realsense SDK 2.0 and OpenCV

[ABaye1]

Hello

I'm not sure if the problem comes from the Intel Realsense library or from OpenCV:

I have a simple program that just gets a RGB image from the Intel Realsense D435, converts the image to BGR with the OpenCV function cvtColor and shows it with OpenCVs imshow.

This works until I activate the depth camera and the RGB camera of the Intel Realsense D435 at the same time. Then I get some frames where it seems like the conversion of the RGB image was not done by OpenCV. So red appears blue and vice versa. This happens in irregular intervals.

I based the code on the OpenCV imshow example from github.

I'm using the newest OpenCV version 3.4.3 and Intel Realsense SDK 2.0 version 2.16.0.

I also posted this question on answers.opencv.org because I'm not shure if this is a problem with Intel Realsense or OpenCV.

Here is my code:

# include <</span>librealsense2/rs.hpp> // Include RealSense Cross Platform API 

# include <</span>opencv2/opencv.hpp> // Include OpenCV APIint main(int argc, char * argv[]) try{ // Declare RealSense pipeline, encapsulating the actual device and sensors rs2::pipeline pipe; rs2::config cfg; //depth stream config cfg.enable_stream(RS2_STREAM_DEPTH, 1280, 720, RS2_FORMAT_Z16, 30); //colour stream config cfg.enable_stream(RS2_STREAM_COLOR, 1280, 720, RS2_FORMAT_RGB8, 30); // Start streaming pipe.start(cfg); const auto window_name = "Display Image"; cv::namedWindow(window_name, cv::WINDOW_AUTOSIZE); while (cv::waitKey(1) <</span> 0 && cvGetWindowHandle(window_name))...

[idata · Accepted solution]

Hello Honk5000,

Thank you for your interest in the Intel RealSense D435 camera. The problem we have noticed is that the source Mat and the destination Mat are the samecv::cvtColor(image, image, cv::COLOR_RGB2BGR);Change the code as below and it will work:

cv::Mat image(cv::Size(w, h), CV_8UC3, (void*)colour.get_data(), cv::Mat::AUTO_STEP);

cv::Mat dst; //added this

//convert from RGB to BGR

cv::cvtColor(image, dst, cv::COLOR_RGB2BGR); //changed destination matrix to dst

// Update the window with new data

imshow(window_name, dst); //show the new destination matrix

Best regards,Eliza