d435 opencv glitch

[PManc1]

Hi, I'm using a d435 and I'm trying to flip the depth image with opencv, but I often get a glitch (the image looks flipped for a dozen of frames and not flipped for a few frames).

If I use recorded bitmap depth the glitch doesn't occur.

This is the code I'm using:

rs2::colorizer c; rs2::context ctx; // Create librealsense context for managing devices rs2::colorizer colorizer; // Utility class to convert depth data RGB colorspace std::vector<rs2::pipeline> pipelines; for (auto&& dev : ctx.query_devices()) { const char* cameraserial = dev.get_info(RS2_CAMERA_INFO_SERIAL_NUMBER); rs2::pipeline pipe = rs2::pipeline(ctx); rs2::config cfg; cfg.enable_device(cameraserial); cfg.enable_stream(RS2_STREAM_DEPTH, COLS, ROWS, RS2_FORMAT_Z16, 30); pipe.start(cfg);   pipelines.emplace_back(pipe);   auto profile = pipe.get_active_profile(); std::ifstream t("test.json"); std::string str((std::istreambuf_iterator<char>(t)), std::istreambuf_iterator<char>()); rs400::advanced_mode dev = profile.get_device(); dev.load_json(str); } std::map<int, rs2::frame> depthframes; while (true) { cv::Mat depth;   std::vector<rs2::frame> new_frames; for (auto &&pipe : pipelines) { rs2::frameset fs; if (pipe.poll_for_frames(&fs)) { for (const rs2::frame& f : fs) new_frames.emplace_back(f); } } for (const auto& frame : new_frames) { c.set_option(rs2_option::RS2_OPTION_HISTOGRAM_EQUALIZATION_ENABLED, 0); rs2::frame v = c.colorize(frame); depthframes[frame.get_profile().unique_id()] = v;   } cv::Mat depth = cv::Mat(cv::Size(COLS, ROWS), CV_8UC3, (void*)((depthframes.begin())->second)); cv::flip(depth, depth, -1); cv::namedWindow("depth"); cv::imshow("depth", depth); cv::waitKey(1);   }

Can someone help me?

Thank you very much.

[MartyG · Accepted solution]

There is a phenomenon where the camera can have the same frame number and timestamp for a few frames. This can occur when a frame has been dropped, and the camera tries to compensate by returning the last known frame. Dorodnic the RealSense SDK Manager has advised that in such cases, "If you are building the SDK from source, please make sure to build with -DCMAKE_BUILD_TYPE=Release. You can also try reducing the total resolution".

If your unflipped frames have the same frame number and timestamp, that may indicate that there is a hiccup and recovery occurring due to dropped frames.