d435 poor depth data quality

[GVign]

Hi to everyone here

we are a little software house and untill last year we were using kinect sensor to build interactive games.

The kinect now is out of stock so we need to change our sensor camera. Intel realsense D435 was, on the paper, a best of a sensor.

The sensor is arrive, I downloaded the software and the SDK and I started to test it and develop or new test software.

The depth quality is worse than the kinect.

I know that the kinect is a differnt kind of sensor, but as you can see from the following pic the point clound mesh has a very low quality.

The big yellow rectangle on che center of the left image is a panel, where we want to detect movements. The righ part of the image is the same object in 3d prospective.

The mesh isn't smooth, we can use the depth data the way they are.

I have captured the image with the intel viewer softare, and event with the post processing filter the quality is not enough accurate.

We need to detect movements and object in a range from 2.2meters to 6 meters. In the image the vertical panel is about 2.5 meters away from the sensor... and I can't use my detecting algoritm.

is the device broken? Or there is a way to improve the quality of the depth data?

[MartyG]

There have been some reports of users not being able to attain the same depth quality on D435 that they had on Kinect. So it is not just you.

In cases where you are getting a black area on the left side of the screen, an initial method you can try in order to get more detail is de-selecting the 'Enable auto-exposure' option in the SDK Viewer program and instead setting gain and exposure manually with sliders.

[GVign]

Is there a possibility that some future firmware updates will improve the quality of the data?

I saw that my device has 5.08 firmware version, and on the intel website is available the 5.9 version.

is there any news about the qualty of the new firmware?

I tryed to play with the auto exposure and with other parameters. I even reduced the frame rate, used my algorithm to reduce the frame rate to 60 to 30 and using 2 frame to create one average frame, but the result isn't soo good. Not as good as we need to our job. And I can't go under 30 fps because I need that frame rate to have that minimum speed to interact with our games/app.

From this page

https://www.intel.com/content/www/us/en/support/articles/000026260/emerging-technologies/intel-realsense-technology.html Depth Resolution of Intel RealSense Depth Camera D435 and Intel...

"...The depth accuracy of the D400 series cameras is less than one Percent of the distance from the object..."

but I'm not even near that kind of result... and I'm really worried about that...

Is this behaviour an intrinsic features of the sensor? Will it be improved?

[MartyG]

Reportedly, users of the D415 model attain better image results. This may be a consequence of hardware differences, as the D415 has its RGB sensor integrated into its circuit board, whereas the D435 model uses a connector to attach the sensor. There's no information yet though to confirm whether the way in which the RGB sensor is integrated on each model has a bearing on image quality.

There is nothing in the latest firmware that would address your image issues. However, the firmware's release notes do give the advice that if you have speckles on your infrared, you may be able to reduce them by reducing the 'laser power' setting.

[GVign]

Tryed to change the laser power. Still not better result.

And if the sensor is not broken or defective I think I will not be able to use the D435 to my goals. :\

I'm very sad about it... it's so fast an small...

[MartyG]

You are using SDK version 2.10.0. I would advise updating to 10.0.1, and then updating to the latest camera firmware version 5.9.2.0 if an SDK update does not improve your image.

SDK

https://github.com/IntelRealSense/librealsense/wiki/Release-Notes Release Notes · IntelRealSense/librealsense Wiki · GitHub

Firmware

https://downloadcenter.intel.com/download/27522/Latest-Firmware-for-Intel-RealSense-D400-Product-Family?v=t Download Latest Firmware for Intel® RealSense™ D400 Product Family

[PSnip]

Hi Marty,

There have been lot of developers who found D415 to be better than D435 in terms of accuracy. This is specially true for Smaller distance, and with smaller objects.

I don't think having a sensor connected directly to the board, or having it connected via a connectors makes any difference. If link from sensor to processor is not stable, the connection will fail due to MIPI errors. So we can't suspect any errors.

However, note that D415 & D435 use different IR sensors.

D415: OV2740, 1920x1080, Diagonal FOV 77, Rolling shutter

D435: OV2982, 1280x800, Diagonal FOV 90, Global shutter

So, as you see the biggest difference is that D435 has smaller resolution, and higher FOV. So spatial context covered by one pixel on D435 is much larger than D415. I feel that this may be the main reason behind drop in performance. However, D435 is claimed to be more suitable for Mobile Robots due to Global Shutter. It also has higher FOV, so may be useful for some specific tasks.

[PSnip]

Hello,

There are many developers who have not been very happy with D435 quality.

One of the suggestions I came across recently was to try with different presets. If you are using Realsense Viewer, then you should try with various presets. There are some more presets which are not available under realsense viewer, but can be tested in custom applications.

You can find information about various presets here

https://github.com/IntelRealSense/librealsense/wiki/D400-Series-Visual-Presets D400 Series Visual Presets · IntelRealSense/librealsense Wiki · GitHub

Apart from Presets, you can also play around with post-processing.

There are some more suggestions on this thread regarding how to improve D435 performance by tuning various knobes.

https://github.com/IntelRealSense/librealsense/issues/1053 https://github.com/IntelRealSense/librealsense/issues/1053

With my personal testing so far, I have seen that D435 can some times fail on surfaces without texture. This happens despite using active IR, which is strange. On some other occasions, D435 can work well under similar test conditions.

Regards,

PS

[MartyG]

Thanks so much for the detailed feedback and suggestions, PS.

[idata]

First off, I'll share my experience: Regarding depth reproduction, not considering cost or form factor, I have a 415 next to a 435 and we're A/B testing them, and we can't find any instance where the 415 outperforms the 435. We have not, however, tried anything that's near-field (including the "hand" preset), we're testing from 1-2 meters out, for the time being, to try to get the sensor under control before we widen our depth range.

Next, beyond the presets, we have not had any meaningful output from adjustments to depth sensor settings regarding enhanced fidelity. And I'd like to clarify that the calibration application has no bearing on the depth reproduction, is this correct? I believe that use of the calibration tool is to set the color camera's distortion coefficients (the K's), the impact of which is limited to the color to depth alignment.

Any confirmation or feedback on the above is much appreciated!

Thx!

[PSnip]

Hi,

In order to check if something is wrong with your D435 module, you can play around with rs-depth-quality tool, if not done already.

Recently Marty shared a link for white paper on depth quality tool

https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/RealSense_DepthQualityTesting.pdf https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/RealSense_Dep…

and this is the git hub link to this tool

https://github.com/IntelRealSense/librealsense/tree/master/tools/depth-quality librealsense/tools/depth-quality at master · IntelRealSense/librealsense · GitHub

I hope using the above tool you can figure out if your module itself is broken, or if this is the kind of performance Intel claims.