What do I need to do object recognition?

[AKarl2]

I'am in a project where we would like to recognize different objects to determen what material it is made of.

We got an R200 camera but I an SR300 is ordered as it seems to be prefered.

There is lot of examples where regular webcameras are used like opencv and SentiSight SDK, but I'am having touble to find examples or SDKs where RGB-D cameras are used which I think is to prefer as the depth gives you extra information about the object.

Any suggestions what I should look for or where to look?

[MartyG]

In 2015 Intel partnered with a Chinese company called JD to trial the RealSense camera's use in warehouse management to analyze box sizes and work out how much space was needed for the boxes. This demo may provide you with useful inspiration.

https://www.youtube.com/watch?v=8lh2EiwQjJ8 IDF Shenzhen 2015 Intel RealSense Technology Partnership w/JD.com - YouTube

[AKarl2]

Maybe I was little unclear about my question. What I'am looking for is a SDK that help me create a application that can determen what material a object is made of by just recognizing the object.

In the opencv example you have to take multiple images of several brands to recognize it's a plastic bottle. But with the depth information and a 3d-scanned bottle to compare with only one 3d scanned model is needed to compare the shape of the object.

What I understand there should be a object recognition sample for the R200 camera in the Realsense SDK, but I can't find it.

[MartyG]

I understood the question. I was primarily quoting the YouTube video as an example of RealSense's depth perception capabilities, Yes, RealSense is capable of looking at an object and recognizing it.

In the Sample Browser application that is packaged with the SDK, you can use the menus on the top and side of the window to filter by camera type (R200) and the language that the sample is in (e.g C++). There is an Object Recognition sample there. It is also available in the C# language.

[AKarl2]

I have now done a clean installation with Realsense 2016 R2 SDK (Lates version for R200), but I can't find the Sample Browser. I had the browser installed when I ran 2016 R3 SDK, but with R2 I can't find that browser.

I read somewhere that R200 is not supported by 2016 R3 SDK that I think would explain why I couldn't find that Object Recognition sample in the first case, when I installed 2016 R3 Essentials SDK.

That Object Recognition sample is found under the R200 Samples tab. Does it work with SR300 too?

[MartyG]

The Sample Browser is typically installed on the desktop, separate from the RSSDK folder, in a folder called 'Intel RealSense SDK Gold'. It can be quite easy to miss if you have a crowded desktop.

The samples can also be accessed directly from the RSSDK/bin/win32 (for 32-bit versions) or RSSDK/bin/x64 (for 64-bit versions for Windows 10). You should ideally copy the sample files to another directory first before using them, since the bin directory is read-only. You do not need to do this if accessing them through the Sample Browser application.

The SR300 does not seem to have an Object Recognition sample, just Object Tracking.

[AKarl2]

Hi again,

Thank you very much for such fast answer. However my SDK was installed by default in c:\ProgramData which looks odd to me. Maybe I got wrong SDK because I can't find any RSSDK folder more than c:\ProgramData\Intel that only holds the Store and v10 folder.

The installationfile was named intel_rs_sdk_runtime_10.0.26.0396.exe

If I only could find the right sdk to install to start with. Wherever I go, I get R3 Essentials that is not supported by R200.

Okey, there is no Object recognition sample for SR300, but the camera supports Object Recognition I hope? Why doesn't the sample support SR300? The app should work the same independent of what rgb-d camera that is used. If it is a limitation in the SDK, I have to look somwhere else for a SDK with Object Recognition that supports SR300.

[MartyG]

Previous SDKs can be downloaded from the link below. The R2 you want is the '2016 R2' at the bottom of the list of download links.

https://software.intel.com/en-us/articles/previous-intel-realsense-install Previous Intel® RealSense™ Install Software | Intel® Software

The 2016 R2 documentation web-page says "The Object Recognition module is in preview quality and works for the camera model R200 only." If I recall correctly, the R200 cam and the cam used for the F200 / SR300 are not the same camera hardware and handle infra-red differently.

Edit: I looked up the version number quoted and it is R2, suggesting you already have the R2 SDK installed.

[AKarl2]

That was the one I downloaded. I finaly found the "real" 2016 R2 SDK. You can donwload it here: https://software.intel.com/en-us/blogs/2016/05/21/new-r2 https://software.intel.com/en-us/blogs/2016/05/21/new-r2

However that object recognition seams to be only on already programmed objects like sofa. I would like to 3d scan my objects and the let the camera recognize them.

So if the SDK doesn't support Object Recognition for SR300. What SDK should I then use with that camera?

[MartyG]

As well as object recognition, RealSense can also 3D Scan any object. In the samples, there is a sample application called '3D Scan' in the C# language that is listed under the 'Common' category of the sample browser, meaning that it should work with any of the cameras. Many people have used RealSense to scan real-world objects, both ones that are static and ones rotating around on a motorized stand.

Here's a real-time scan of me using the application that was taken with my F200 camera (the predecessor of the SR300).

Edit: regarding recognizing object once scanned: although you can 3D-scan with the SR300, object recognition seems to be limited to the R200.

[AKarl2]

I tried the Object tracking with the SR300 camera, and it is almost what I need for the project. I loaded a image of a object and the software recognized the object. But if I turn the back of the object it will offcourse not be recognized, and that is what I hope a 3d scanned object would help me with instead of multiple images of every object.

The 3d information would also make the reconization more trustfull. Is there a good SDK free or commercial out there that can help me with this?

I was looking at opencv, but I can't find any application that does that 3d object recognization with opencv. Maybe someone here knows better?

[MartyG]

Other camera SDKs such as Librealsense and OpenCV are sadly outside of my field of knowledge, but I'm sure someone here will have some good advice. Good luck!

[idata]

Hi guys,

First of all I want to say thanks to you guys, you made this thread very interesting and helpful to the community. Now as a complement to Ankan, you can check SDK R3 3D-Scan documentation here: https://software.intel.com/sites/landingpage/realsense/camera-sdk/v2016r3/documentation/html/index.html?doc_scan_3d_scanning.html, maybe you can find something helpful.And we encourage you to continue with this interesting project to make it work, and feel free to share it here to help to community.Have a nice weekend.Regards,-Leonardo