Solved: Help needed with declaring and applying post-processing filters via code (Visual Studio C++ with OpenCV using the D415 RealSense)

KChow4 · ‎01-07-2019

I used the following code (C++ in Visual Studio using OpenCV and RealSense D415) to try and use the post-processing filters without using the whole UI thing in the tutorial, and it does not look like the filters are actually applying onto the image at all.

Is it possible to simply call up filters like I am trying to do? And if so, where am I going wrong in my current code?

Thanks in advance!

//Testing code for filtering images via accessing post-processing filters
 
#include "pch.h"
#include <iostream>
#include <librealsense2/rs.hpp> // Include RealSense Cross Platform API
#include <librealsense2/rs_advanced_mode.hpp> //Include RealSense Advanced Mode
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/opencv.hpp>   // Include OpenCV API
#include <string>
 
using namespace std;
using namespace cv;
using namespace rs2;
using namespace rs400;
 
int main(int argc, char * argv[]) try
{
	string serial_no = "123123412312345";  //In case anyone is wondering, I have just put a dummy serial number here as a placeholder
 
	//Used to obtain a list of all devices currently present
	context ctx;
	auto devices = ctx.query_devices();
	size_t device_count = devices.size();
	//If there is no device_count, then there is no connected camera or there is an issue (Print error message)
	if (!device_count) {
		cout << "No devices found." << endl;
		return EXIT_SUCCESS;
	}
	//Get the first connected device (since we probably only have 1 attached)
	auto dev = devices[0];
 
	// Declare depth colorizer for pretty visualization of depth data
	rs2::colorizer color_map;
	color_map.set_option(RS2_OPTION_VISUAL_PRESET, 1.f); 
	color_map.set_option(RS2_OPTION_HISTOGRAM_EQUALIZATION_ENABLED, 0.f);
	color_map.set_option(RS2_OPTION_COLOR_SCHEME, 2.f);
 
	// Declare RealSense pipeline, encapsulating the actual device and sensors
	rs2::pipeline pipe;
 
	rs2::config cfg;
	cfg.enable_device(serial_no);
 
	//Enter the advanced mode functionality
	auto advanced_mode_dev = dev.as<rs400::advanced_mode>();
	//Declaration to enable control over the depth table and to get its current settings
	STDepthTableControl depth_table_control_group = advanced_mode_dev.get_depth_table();
 
	depth_table_control_group.depthClampMin = 0;
	depth_table_control_group.depthClampMax = 589;
	depth_table_control_group.depthUnits = 1000;
 
	advanced_mode_dev.set_depth_table(depth_table_control_group);
	pipe.start(cfg);
 
	rs2::decimation_filter dec_filter;
	rs2::spatial_filter spat_filter;
	rs2::temporal_filter temp_filter;
 
	//Declaration of frameset
	rs2::frameset data;
	for (int i = 0; i < 30; i++) {
		data = pipe.wait_for_frames(); // Wait for next set of frames from the camera
	}
 
	//Get the depth map
	rs2::frame depth = data.get_depth_frame().apply_filter(color_map);
	
	dec_filter.invoke(depth);
	depth = dec_filter.process(depth);
	spat_filter.invoke(depth);
	depth = spat_filter.process(depth);
	temp_filter.invoke(depth);
	depth = temp_filter.process(depth);
 
	// Query frame size (width and height)
	const int w = depth.as<rs2::video_frame>().get_width();
	const int h = depth.as<rs2::video_frame>().get_height();
 
	// Create OpenCV matrix of size (w,h) from the colorized depth data
	Mat image(Size(w, h), CV_8UC3, (void*)depth.get_data(), Mat::AUTO_STEP);
 
	namedWindow("cap", WINDOW_NORMAL);
	resizeWindow("cap", 960, 540);
	imshow("cap", im_inv);
 
	waitKey(0);
 
	return EXIT_SUCCESS;
}
 
catch (const rs2::error & e)
{
	std::cerr << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n    " << e.what() << std::endl;
	return EXIT_FAILURE;
}
catch (const std::exception& e)
{
	std::cerr << e.what() << std::endl;
	return EXIT_FAILURE;
}

jb455 · ‎01-07-2019

I had problems implementing the post-processing stuff like that too; I had better success using a custom processing block like in https://github.com/IntelRealSense/librealsense/blob/master/examples/measure/rs-measure.cpp#L191

View solution in original post

MartyG · ‎01-07-2019

Official documentation for post-processing, with a C++ example script at the bottom of the page, can be found at the link below.

https://github.com/IntelRealSense/librealsense/blob/master/doc/post-processing-filters.md

I am not speciaized enough in RealSense programming to diagnose your scripting, regretfully, but I hope that the link above helps you.

The RealSense developer UnaNancyOwen, who has produced a range of OpenCV-equipped RealSense sample programs, also has one for post-processing filters.

https://github.com/UnaNancyOwen/RealSense2Sample/tree/master/sample/Filter

jb455 · ‎01-07-2019

I had problems implementing the post-processing stuff like that too; I had better success using a custom processing block like in https://github.com/IntelRealSense/librealsense/blob/master/examples/measure/rs-measure.cpp#L191