How can I perform collision avoidance with a d435i camera?

[AMaqs1]

I am working on a project where I need to perform collision avoidance with a robot using the D435i camera. I saw a youtube video of intel where you guys performed the same task on a robot.

Can you please guide me with the steps to perform the project or any guideline or tutorial for the steps that you took while doing that project.

Thanks

[MartyG]

Could you provide a link to the video that you are referring to, please? Thanks!

[AMaqs1]

Below are the two links to the video which performed the same task that I want too. However, for initial stage I only want to use the depth camera.

https://www.youtube.com/watch?v=LBMsWJJxLXQ

https://www.youtube.com/watch?v=H2zgUci2ZDE

[MartyG]

I thought that they were likely the ones that you meant but wanted to make sure. Thanks!

As those two examples use the T265 Tracking Camera, if you only have the D435i depth camera then Intel's guide to doing SLAM navigation for D435i using ROS may be a better starting-point.

https://github.com/IntelRealSense/realsense-ros/wiki/SLAM-with-D435i?language=en_US

Intel also has a ROS package for intelligent object analytics.

https://github.com/intel/ros_object_analytics

[CGupt2]

Could you please provide some steps for doing obstacle avoidance using T265 and D435 cameras? Thanks!

[AMaqs1]

Thanks for the reply.

So, you mean to say if I am trying to do obstacle avoidance with only d435i camera, then SLAM Navigation should be the first step to start with?

Secondly, if I also order a T265 tracking camera, then what are the steps that I need to follow to design the same system as shown in the video? Are the built in packages available for the steps to run directly?

[MartyG]

To do obstacle avoidance with D435i, a good option may be an open-source drone flight control software called Dronecode PX4. It features instructions for use with D415 or D435 cameras.

https://forums.intel.com/s/question/0D70P000005SLIYSA4

That page also features a link to a RealSense user who was using object avoidance with D435 on a drone with a ROS-based setup. They provide a short demo video and a ROS launch file

https://forums.intel.com/s/question/0D50P0000490XM0SAM/noise-in-realsense-d435-pointcloud

Regarding the T265 videos, Intel are adding to the example programs and tutorials available for it as an ongoing process. Tracking-compatible examples are listed in the link below.

https://github.com/IntelRealSense/librealsense/blob/master/examples/tracking.md

[AMaqs]

I read the links that you provided. But I just want to do obstacle avoidance with the d435i camera without using any other additional board and regarding the second link, where ROS-based setup is used, i am not using the ROS setup.

I saw your following message in the links provided,

"If you just want to detect the depth of an object on the image and get a value for its distance from the camera, the RealSense SDK documentation has an example script on its front page.

https://github.com/IntelRealSense/librealsense#ready-to-hack

It would be possible to expand upon that script to do an 'If' logic statement that checks the distance value and IF it is a certain distance or less from the camera, trigger the drone to stop."

Can you guide me how to access the ready to hack code?

[MartyG]

To run the Ready To Hack script, you would need to build it into a runnable program using a compiler such as Microsoft Visual Studio or similar software. Dorodnic the RealSense SDK Manager provides advice about setting up Visual Studio for creating a RealSense project in the link below.

https://github.com/IntelRealSense/librealsense/issues/1196#issuecomment-367305613

[AMaqs]

Thanks for the reply.

The steps explained in the link are for the windows. Is there any similar link where the steps to run the script on Linux are mentioned?

[MartyG]

The process of compiling scripts on Linux is not something that I personally have direct experience with, unfortunately. Intel's Linux installation documentation page has detailed instructions about setting up a Librealsense installation on Linux. I hope that will provide some useful guidance for you.

https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md

[AMaqs]

I have already installed the software on Linux. I just want the guidance to run the ready-to-hack script on Ubuntu 16.04.

[Eliza_D_Intel]

Hello AMaqs1, The "Ready-to-Hack" needs to be used, as MartyG previously mentioned, with a compiler of choice. Unfortunately, we are unable to provide instructions for each compiler. Could you please point out what is the video you have watched? Thank you, Eliza

[AMaqs]

Thanks for your reply.

What I meant to ask was how to run the ready-to-hack script on linux. The compiler doesn't matter as I am flexible with it but I need the instructions to run the script on ubuntu.

Secondly, I was talking about the following two videos,

https://www.youtube.com/watch?v=LBMsWJJxLXQ

https://www.youtube.com/watch?v=H2zgUci2ZDE

However, in this video, intel has performed the localization and mapping too but my task is to simply avoid the obstacles that are in the path of the robot.

For example, when a robot is running, it detects the obstacle in its path and avoid the collision with it. No need to map the environment.

[Eliza_D_Intel · Accepted solution]

Hello AMaqs1, Please take a look at the description of the first link: "We will be releasing some of the code used in this demo in the near future on our github". Please keep an eye on https://github.com/IntelRealSense/librealsense for when it will be released. For the second link, we cannot comment about the code, as it was designed by Embedded Vision Alliance. You can reach them out for further instructions/source code: https://www.embedded-vision.com/ As for the compiler, MartyG posted a GitHub issue with instructions from RealSense Team on how to compile codes for Visual Studio. You will need to port the solution yourself for the compiler of choice on Linux. Thank you, Eliza