Items with no label
3335 Discussions

How can you find multiple objects using "SR300"?

승노
Beginner
1,848 Views

I want to find several objects. So I actually found a 3D object using "The Metaio toolbox". However, there is a problem with finding multiple objects that is not well recognized. How can you find multiple objects using "SR300"?

0 Kudos
9 Replies
MartyG
Honored Contributor III
691 Views

In the old 2016 R2 SDK, the SR300 could do object tracking, but object recognition was only for the R200 model of camera.

The current RealSense SDK 2.0, which is compatible with the SR300, can do object recognition though with the help of the OpenCV software. Here's a couple of example programs.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/opencv/dnn librealsense/wrappers/opencv/dnn at master · IntelRealSense/librealsense · GitHub

https://github.com/twMr7/rscvdnn GitHub - twMr7/rscvdnn: Test program for OpenCV DNN object detection with RealSense camera

0 Kudos
승노
Beginner
691 Views

I'm doing something similar to the example "object tracking." But in my program, it's well recognized for a single object. However, if you have multiple objects to recognize, it is not well recognized.

The parts that represent the objects being tracked are:

private void UpdateObjectTraking()

{

// 추적하고 있는 객체를 표시.

foreach (int id in targetIds1)

{

PXCMTracker.TrackingValues trackData;

var sts = tracker.QueryTrackingValues(id, out trackData);

if (sts < pxcmStatus.PXCM_STATUS_NO_ERROR)

{

continue;

}

if (PXCMTracker.IsTracking(trackData.state))

{

if (trackData.cosID == 1)

{

ShowTrackingValue(trackData); //ShowTrackingValue, it's simply the part where the image is displayed.

}

}

}

foreach (int id in targetIds2)

{

PXCMTracker.TrackingValues trackData;

var sts = tracker.QueryTrackingValues(id, out trackData);

if (sts < pxcmStatus.PXCM_STATUS_NO_ERROR)

{

continue;

}

if (PXCMTracker.IsTracking(trackData.state))

{

if (trackData.cosID == 2)

{

ShowTrackingValue(trackData);

}

}

//else targetBottle = false;

}

}

Can I get some advice?

0 Kudos
MartyG
Honored Contributor III
691 Views

Using your SR300 camera with the OpenCV software is your best chance of achieving multiple object recognition, unfortunately. The object recognition mode in the '2016 R2' SDK was classed as being a 'Preview' state, meaning that it was an early-stage prototype that was not developed far.

0 Kudos
승노
Beginner
691 Views

So how do we find Multiple targets in the following example? Don't you know about this?

0 Kudos
승노
Beginner
691 Views
0 Kudos
MartyG
Honored Contributor III
691 Views

In that sample program, you provide the program with image files to teach it what to look for, and the program then looks for similar-looking areas in the camera's view and highlights them.

So technically, according to the documentation for the program, it can highlight two different objects at the same time if you loaded in two different reference images, and the program can see elements in the camera view that look similar to both of these images.

I do not know if it can recognize two similar-looking examples of the same object, though it probably can. Give it a try.

0 Kudos
승노
Beginner
691 Views

Thank you. We actually configured the program, and we found that the recognition rate has dropped significantly. Is there a way to raise awareness?

0 Kudos
승노
Beginner
691 Views

And it's not well recognized by using the background in bright green and then turning it dark brown. Does the background shade affect you?

0 Kudos
MartyG
Honored Contributor III
691 Views

I cannot see anything in the options to affect tracking performance. It is my experience with older RealSense models such as the F200 and Sr300 though that it is more likely to recognize and track objects that are in the center of the camera's view. Tracking may be lost with objects that are off to the side of the camera view, because those older moels have a smaller field of view (FOV) than the current 400 Series cameras.

Regarding the problems when changing background color: there is a known problem with depth cameras of any type that dark gray or black surfaces are hard for the camera to see, because the laser's light gets absorbed by it. .Maybe you are experiencing a similar absorption with dark brown.

0 Kudos
Reply