D415 - Basic Code For Saving Depth Stream As Binary File

[RAnyo]

I am a complete beginner to coding. I can do some things in Matlab and Python but that's about it. All I want to do is record depth with my D415 camera and save the stream as a binary file (little endian) that contains the depth value of every pixel for each frame (all I want is depth, nothing else). So if my resolution is 840x480, I want a file that has 840 X 480 depth values for every frame. Here is what I can do myself. I can implement, the examples on github. I am able to get my linux machine to run:

$ g++ -std=c++11 rs-save-to-disk.cpp -lrealsense2

from https://github.com/IntelRealSense/librealsense/tree/master/examples/save-to-disk librealsense/examples/save-to-disk at master · IntelRealSense/librealsense · GitHub

or

$ gcc depth.c -lrealsense2

from https://github.com/IntelRealSense/librealsense/tree/master/examples/C/depth librealsense/examples/C/depth at master · IntelRealSense/librealsense · GitHub

My understanding of these languages is so small that I cannot simply read the sdk and do what I want. (If I was working in MATLAB I could do that no problem).

There must be code out there SOMEWHERE that accomplishes my very basic task.

My end goal is to be able to record with the camera for hours. I eventually want to model the depth data and do some real time processing (sort of like object recognition). Please help me out, I'd be very grateful. I'm looking for code I can copy and paste and use right away or a step by step (so simple that a child could understand) walk through of how to do this myself. I'd prefer the former. In the meantime I am learning c and c++ from scratch.

Thanks

[MartyG]

A MATLAB wrapper is on SDK 2.0's list of upcoming features. Hopefully this will make development easier for you, since MATLAB is the environment that you are most comfortable with. EDIT: the MATLAB wrapper is now available from SDK version 2.16.0 onwards.

In the meantime, there is a Python tutorial in the SDK for displaying the depth stream in the console as ASCII text characters.

https://github.com/IntelRealSense/librealsense/blob/development/wrappers/python/examples/python-tutorial-1-depth.py librealsense/python-tutorial-1-depth.py at development · IntelRealSense/librealsense · GitHub

You could perhaps add a line to the script that prints the contents to a text file too. For details of how to do this, google 'python print to text file'. Looking at the example script, it seems that the output is stored in the variable called 'line' and printed to screen.

Here's an example of a guide to adding code to print to a text file.

https://stackoverflow.com/questions/5214578/python-print-string-to-text-file?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa Python Print String To Text File - Stack Overflow

[NTala]

Hello MartyG,

I tried to implement something in MATLAB which is analogous to the python link you pointed to however, I notice that reading depth through the function call depth.get_distance(x,y) is taking quite some time when trying to get depth for the entire field of view. Specifically, I notice that it takes about ~9 seconds to get depth of every pixel in a given frame using the depth call. I am using RealSense d435 with 1280*720 resolution and FPS = 30. The code used is pasted below:

%Make Pipeline object to manage streaming

pipe = realsense.pipeline();

% Make Colorizer object to prettify depth output

colorizer = realsense.colorizer();

%Start streaming on an arbitrary camera with default settings

profile = pipe.start();

while (true)

fs = pipe.wait_for_frames();

depth = fs.get_depth_frame();

tic

for i = 1:720

for j = 1:1280

dist(i,j) = depth.get_distance(j,i);

end

end

toc

color = colorizer.colorize(depth);

data = color.get_data();

img = permute(reshape(data',[3,color.get_width(),color.get_height()]),[3 2 1]);

imshow(img);

end

Is there a faster way to do this so that I get depth value for the entire FOV under 1s for example? Maybe, the way it is done on the RealSense viewer perhaps? (which is displaying a video at 30 FPS while giving me depth info when i hover over a pixel)

Your help is much appreciated!

Regards,

Neel

[MartyG]

If you want to display the depth frame in MATLAB, the recently introduced MATLAB wrapper for SDK 2.0 has an example script for doing so on the front page of its documentation. You'll find it if you go right to the bottom of the page.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/matlab librealsense/wrappers/matlab at master · IntelRealSense/librealsense · GitHub

Edit: the wrapper requires Windows 10. Do you have Windows?

[NTala]

Thanks for the quick response!

Yes, I had looked at it earlier however, that example is not giving me physical distance for each pixel in its FOV. The depth example seems to be only displaying one depth frame. I am looking to get a depth value (in m) for each pixel in the FOV of the camera and store it in a matrix and do it repeatedly for every frame. I am currently able to do that but it is taking very long. I am wondering if there is a faster way to do it.

Edit: Yes I have windows 10

Regards,

Neel

[MartyG]

What about the Python distance_to_object tutorial that measures depth?

https://github.com/IntelRealSense/librealsense/blob/jupyter/notebooks/distance_to_object.ipynb librealsense/distance_to_object.ipynb at jupyter · IntelRealSense/librealsense · GitHub

Edit: no, it has to involve MATLAB, doesn't it. Sorry. Checking again ...

Edit 2: I recalled that Dorodnic the SDK Manager gave another user a MATLAB depth script for Pyrealsense today.

https://github.com/IntelRealSense/librealsense/issues/2520# issuecomment-429339555 https://github.com/IntelRealSense/librealsense/issues/2520# issuecomment-429339555

In that script, it is setting the colorizer configuration to 'White to Black' instead of the 'Jet' setting. So you may have to change the setting number to the configuration you want.

[NTala]

Thanks for all the help!

So the links you posted were both for python and I am working in MATLAB, but they were still very helpful! As it turns out, I found the magical line of code I was looking for which can basically give me depth for every pixel in the FOV in a single call!! This makes everything a lot faster. For future reference to others, the following lines of code will provide depth (in mm) in a row vector form, for every pixel in a frame, using a single call.

pipe = realsense.pipeline();

fs = pipe.wait_for_frames();

depth = fs.get_depth_frame();

dist = depth.get_data();

Cheers,

Neel

[MartyG]

When you said you were trying to create something analogous to the Python link, I thought you were using Python and MATLAB together. Apologies for the confusion.

Awesome news that you found a solution. Thanks for sharing your code with the community!