Feature tracking and/or optical flow with the R200?

[TKirv1]

Would it be practical to do feature tracking or optical flow using the R200 points data stream, or this something that could be done on the camera module possibly using the ASIC? What exactly does the ASIC compute? I read that it determines the depth values for each pixel, but I was wondering if that was all it does (not to say that isn't impressive)?

Thanks,

Thomas

[MartyG · Accepted solution]

Page 23 of the R200's data sheet says "The R200 ASIC is a USB3 composite device which exposes all hardware endpoints to the operating system. The ASIC is a bulk device and transmits depth and color videos streams in data bursts rather than as constant video streams."

The R200 data sheet:

https://software.intel.com/sites/default/files/managed/d7/a9/realsense-camera-r200-product-datasheet.pdf https://software.intel.com/sites/default/files/managed/d7/a9/realsense-camera-r200-product-datasheet.pdf

My research into the subject of optical flow suggested that the R200 would struggle to cope with it on its own. On RealSense-equipped drones, the R200 is used in combination with a pair of VGA cameras. An Intel article suggests that a use for the VGA could be optical flow.

https://software.intel.com/en-us/articles/intel-aero-compute-board-and-intel-realsense-technology-for-wi-fi-streaming-of-rgb-data RealSense - Intel® Aero Compute Board and Intel® RealSense™ Technology for Wi-Fi* Streaming of RGB Data | Intel® Softwar…

[TKirv1]

Thanks for the reply MartyG,

Interesting. I had no Idea this product existed. Initially I was planning on using just the R200 to do collision avoidance with a quadrotor I built, but now that I see this I am wondering if it would be better.

Would you recommend trying to integrate this kit with something other than the Aero Compute module, say the UP-board?

Thanks,

Thomas

[MartyG]

As it's classified as a drone-related product, it's in the drone kit section of the Intel online store instead of the RealSense developer kit section, so it is easy for RealSense developers to miss unless they browse around.

There are similarities between the Aero Compute board and the Up Board, in that they both run Linux (Aero uses a flavor of Linux called Yocto Project, and Up uses Ubuntu) and they both use an Atom processor. I assume there is a good reason why the Intel drone kit uses its own specialized Aero board instead of an Up board though. For example, the Aero board info says that the board has "reconfigurable I/O to facilitate connecting to a broad variety of drone hardware subsystems."

The Intel drone kit aims to make it relatively straightforward to build a drone using a set of recommended components that are known to work together (drone, Aero board, Vision Kit, etc). I would imagine that it would be possible to integrate a different control board though so long as it was R200 compatible, if you have the drone building skills to do so.

As an example, there is the early RealSense drone from 2015 with six R200 cameras in a 360 degree arrangement connected to a single onboard board.

http://bit.ly/2jVQjVG http://bit.ly/2jVQjVG

[TKirv1]

You would think there would be many subsystems needed in a drone, but I was able to build one that does position hold and command following using two sensors, an Intel Edison, and an external PWM driver. Amazingly the ONLY hardware subsystem needed from the Edison is the I2C bus. The reason I am switching to the UP-board is because the thing needs to be mostly autonomous so I need a capable forward facing depth camera (Realsense -> usb 3.0 -> up-board).

I was wondering: Since the drone kit makes it relatively straightforward to build a drone, does this mean it has a pre-programmed flight controller? or can you program any flight controller you'd like?

Thanks again for the info! I will look further into the using the vision Kit with the up-board.

Thomas

[MartyG]

My understanding from customer feedback on Intel's 'Ready To Fly' drone kit is that whilst it makes drone building easier, you still need a certain amount of drone building and flying skill.

According to the Ready To Fly drone's store listing, its flight controller can use Dronecode PX4 autopilot.

https://github.com/PX4 PX4 Pro Drone Autopilot · GitHub