Displaying features of SR300 simultaneously

[AHase]

Hi Guys,

I'm totally new to this field so this question may sound strange but here we go:

Is it possible to display various features like object recognition, person tracking, SLAM and gesture recoignition (additionally the raw data/video stream too) at the same time on different screens?

From my understanding what I learned during my research it should be possible and only limited by the hardware resources.

Because the camera only sends video streams and the functions for the features run on the machine the camera is connected to and not the camera itself.

Is my assumption correct?

At the end it should run on Ubuntu 16.04 with ROS Kinetic.

Don't know what hardware is need for this except USB 3.0 as interface for the camera...

Sincerely, Alex

PS: I hope the thread/question is in the correct categorie

[MartyG]

May I confirm please that when you say you want to display features on different 'screens', you mean different windows on the same computer screen?

[AHase]

@MartyG:

Unfortunatly no, I meant literally different computer screens...

And there is no way around that :/

[MartyG]

Okay, thank you. And are the extra display screens attached to the same computer, or do you want to display them on the screen of a different computer?

[AHase]

This part isn't elaborated right now.

But I would say on the one hand it would be nice if I needed only one computer to do the task... on the other the costs shouldn't be too high.

In short: the plan was to build a driving robot and on the robot is a frame where we want to visualize (different) sensors and a 3D-camera is one of it.

The displays should be on all four sides and as you may have guessed by now: there is no place for big computers.

Something small like single-board computers (is that the correct term?).

So I would say the plan with the screens is quiet fix, but the hardware topology between camera and screens is not.

For the hardware it really depends what suits best without loosing sight of the costs.

[MartyG]

If the duplicate display did not need to be interactive then you could use some form of "casting" device such as Google Chromecast to send the contents of a screen over wireless wi-fi signal to another screen that has the casting device plugged into a connector on it such as an HDMI port. Google 'stream chromecast from linux' for more details about Chromecast and Linux.

The displays put round the sides with Chromecasts inserted in them could even be ordinary flat-screen TVs instead of computer screens.

[AHase]

Thank you for the hint

And my assumption in the beginning, that the features like person tracking, SLAM etc is "made" on the computer and no the camera, is that correct?

[MartyG]

The camera collects the data, but it is the computer, or a single-board kit computer, that turns the data into something useful.

[AHase]

Thank you very much!

So I just need a single-board kit computer that is powerful enough.

Would be easier than splitting the tasks on different computers I think.

[MartyG]

The 400 Series cameras (D415 and D435) work with any Intel or ARM processor, with the main requirement being a USB 3.0 port. So your possible choice of suitable single-board computers is considerably wider now than it was with earlier RealSense camera models.

You will find it very difficult to find a single-board computer that will work with the SR300 unfortunately, as it prefers an Intel Core processor, which single-board computers tend to lack.

If the camera will be in motion, the D435 is the most suitable of the two, as the D415 excels at static scanning,

[AHase]

First we wanted to get the 400er Series cameras, but shipping time was somewhat around 12 weeks...

But thanks for the hint with "camera in motion"!

[MartyG]

Which country are you located in, please? If you are in a country that is on the list of countries that Intel ships deliveries to then you could get D435s much sooner by ordering from the Intel Click online store. There would still be a wait but much sooner than 12 weeks (maybe sometime around April).

https://click.intel.com/intelr-realsensetm-depth-camera-d435.html Intel® RealSense™ Depth Camera D435 - Intel® RealSense™ Depth Cameras

[AHase]

Germany, I'm gonna talk to my supervisor for this case

EDIT:A question came up... is it possible to do SLAM with the SR300-camera alone?Because to me it doesn't seem so...

[MartyG]

Yes, Germany is on the list of countries that Intel can send the D435 to if it is bought from the Click store. Best of luck!

[MartyG]

You are not likely to get anywhere with SLAM on the SR300, as it is a type of camera known as a Coded Light camera, whereas the RealSense models suited to SLAM are Stereo type cameras like the 400 Series, ZR300 and R200.

The ZR300 cannot be bought from the Intel store anymore as it is a retired product, though some Intel Approved Distributor stockists still have some limited quantities, such as Avnet, who currently have 4 left.

https://www.avnet.com/shop/emea/products/intel/955176-3074457345633931395/ Avnet: Quality Electronic Components & Services

The R200 can still be bought from the Click store as part of the Robotic Development Kit, a bundle deal containing an R200 and an Up Board single-board computer.

https://click.intel.com/intelr-realsensetm-robotic-development-kit-2484.html Intel® RealSense™ Robotic Development Kit - Intel® RealSense™ Depth Cameras

[AHase]

RGB-D SLAM works even with a Kinect, that isn't a Stereo camera.

Or would this method don't work either because the "coded light" thing?

[MartyG]

I use the words "not likely" possible because I know better than to say something is impossible. "As rare as chicken teeth" is the phrase that comes to mind. The only example I know of regarding RGBD SLAM with an SR300 is a paper written about depth calibration with Visual SLAM. Stereo cameras are the most well-supported means of achieving SLAM with RealSense.

https://www.ais.uni-bonn.de/papers/IROS_2017_Quenzel.pdf https://www.ais.uni-bonn.de/papers/IROS_2017_Quenzel.pdf

I believe that the coded light system the SR300 uses is like Kinect 1's. Microsoft switched to a different projection method called time of flight for Kinect 2.

[AHase]

Haha, that's the paper I found

I tought it could work because it is a method that should in general with an RGB-camera with depth sensing.

As for now, I didn't get far enough into the libraries from RealSense, but to me it seams SDK 2.0 has no lib for SLAM... is that correct?

[MartyG]

You can use SLAM with SDK 2.0 by using it in combination with a SLAM system for the OpenCV platform called ORB_SLAM2.

https://github.com/raulmur/ORB_SLAM2 GitHub - raulmur/ORB_SLAM2: Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalizati…

Intel are also working on a new RealSense product called the V200, a headset that is paired with an Intel RealSense SLAM Vision Library. The V200 already has initial support built into SDK 2.0.

https://realsense.intel.com/solutions/# virtual-reality Solutions - Intel® RealSense™

[AHase]

In the spec sheets, the SR300 has a depth vision only for about 1.2 to 1.5 meters.

How far can a D400-series camera sense? I looked into a pretty detailed sheet but didn't find any info...

[MartyG]

The 400 Series can depth scan up to 10 m thanks to its advanced D4 Vision Processor component.