Analyze Depth images in Unity

[MTont]

I just installed the relasense SDK 2.0 on unity. I'm using 2 depth cameras (Intel d415 and Intel d435).

Now, I've the colored depth image as in the examples. How can I analyze this images? I need to find the nearest object and draw a square around it.

Is there something similar to Opencv in Unity? I can't find anything but a plugin with very bad rates...

[MartyG]

Was the plugin that you saw the OpenCV For Unity one?

https://assetstore.unity.com/packages/tools/integration/opencv-for-unity-21088 OpenCV for Unity - Asset Store

If you were able to use OpenCV within Unity then you may be able to draw a box simply by accessing the Rectangle function of OpenCV.

[MTont]

Yes, I see now that the ratings are not so low...but price is a little to high.

I wanted to see if there is something else and then I will try that add-on.

Thank you

[MartyG]

Unity has an inbuilt function called GUI.Box that can draw a box on the screen. The problem would be programming an event to trigger this function, such as when the camera reads the depth of an object.

https://docs.unity3d.com/ScriptReference/GUI.Box.html Unity - Scripting API: GUI.Box

[MTont]

Good! this is interesting, but the main problem remains how to analyze the depth image. I think that the add-on with opencv is the only method...

[MartyG]

As the Unity wrapper is based on C# (the language that Unity uses), if I were writing your program then I would attempt to integrate the basic 'Hello World' C# example for analyzing depth into a Unity script.

***********************

var pipe = new Pipeline();

pipe.Start();

while (true)

{

using (var frames = pipe.WaitForFrames())

using (var depth = frames.First(x => x.Profile.Stream == Stream.Depth) as DepthFrame)

{

Console.WriteLine("The camera is pointing at an object " +

depth.GetDistance(depth.Width / 2, depth.Height / 2) + " meters away\t");

Console.SetCursorPosition(0, 0);

}

}

[MTont]

I tried to modify the "RealsenseDevice" script adding the depth.GetDistance method and it works. I just analyze one pixel every 50 and I misure the distance. the problem is that it isn't very precise. Should I calibrate the camera or something similar?

[MartyG]

RealSense cameras are usually well calibrated and only need re-calibrating if something unusual happens to them or they take a hard knock / drop.

Accuracy may be influenced by how near the camera is to what you are measuring. At very close range, the accuracy of the 400 Series can reduce because of how its pixel scaling is set up to enable it to see a distance of around 65 meters (though depth scanning is limited to 10 meters maximum by default). If you are doing close-range scanning, you can change the camera's depth scale to improve accuracy.

A less complex method of reducing the camera's minimum distance is to reduce the resolution to a low one, less than 848x400.