FPS and reconnection issues

[JM16]

We've noticed two issues during our RealSense tests. We're using 2x D415 and 1x D435 cameras with a C# application in Windows 10. We're using firmware v5.9.13 currently.

1. The latest Intel driver locks our framerate to 12fps maximum. If we use the default Microsoft driver, we can run at (our preferred) 30fps. Is there something that we might be doing wrong when initialising our camera?
2. If we update the camera firmware to the latest (v5.10.13) and reboot our PC, the RealSense cameras are somehow locked and inaccessible via our application until we physically unplug/plug in the devices. If we use the older v5.9.13 firmware, we don't have this issue. This is consistent in both the Intel and Microsoft drivers.

Has anyone experienced these issues before, or have any ideas as to what we might be doing wrong? Could it be changes that were made in the 2.0 SDK that we missed? Thanks!

Edit: Important note - we're interested in a high IR framerate, but less concerned with RGB stream. We're trying to stream IR at 30fps but RGB at 6fps. This looks doable in the RealSense Viewer, but when we try via C# we need to keep both streams at the same rate, otherwise the overall framerate drops?

[MartyG]

If you put a hardware_reset() instruction at the start of your script then it can help to avoid having to do physical unplug-replugs of the camera.

https://github.com/IntelRealSense/librealsense/issues/1086#issuecomment-361176576

Regarding your FPS problems, are you changing the camera's settings in your application whilst it is running, please? Some users have experienced problems whilst doing so, whereas the streams work fine if they do not change the camera settings during runtime.

[JM16]

Thanks for the suggestion! We'll do some tests and see if that instruction helps.

Re: FPS issue, the streams are set at launch/initilalization and not changed dynamically. Doing further tests, I confirmed that if we set the IR to 30fps and the RGB to 15fps, the IR framerate slows down and fluxuates between 15-20fps.

Somewhat related, through testing I also noticed that I cannot set the IR stream to 30fps and the RGB stream to 25fps. These are supported in the official RealSense viewer, however the streams timeout via API. Have you seen this behavior before?

[MartyG]

Are you doing an Align on your two streams? If so, are you using the newest RealSense SDK? The SDK had CUDA optimization for Align operations added in version 2.16.5 for greater align efficiency. cv:align operations still have efficiency issues in Windows 10 though.

Dorodnic the RealSense SDK Manager has said to another user with FPS issues, "Another thing that can affect FPS is auto-exposure of RGB if the scene is dark. You can disable it in the Viewer, just to make sure".

I was not sure how the RGB stream could be set to 25 FPS in the Viewer, as the RGB section's FPS drop-down menu does not have a 25 FPS option. Did you mean 25 FPS on IR and 30 FPS on RGB? Generally, 25 FPS is not meant to be used in applications, as it is a speed intended for use only for calibrating the camera.

[JM16]

Apologies, I was mistaken on stream framerates!

I don't believe the issue is related to autoexposure. To save on overhead, we prefer to manually set the RGB stream very low (6fps) and keep the IR stream higher (30fps). However when doing this via C# API, the IR stream framerate drops. For example, when we set the RGB framerate to 6fps and the IR framerate to 30fps, the camera will stream the IR data at 12fps. If we set the RGB framerate to 15fps and the IR framerate to 30fps, the camera will stream the IR data at ~18fps (I've seen a range of 15-25fps, approximately). We can only achieve 30fps on the IR stream when we also set the RGB stream to 30 fps.

What's odd is that we don't have this issue when we use the default Microsoft imaging driver - only with the Intel driver (currently running the latest version).

[MartyG]

The best place to get an answer on this question is likely to be at the RealSense GitHub website, where the RealSense engineers and developers reside. You can post your problem there by visiting the link below and clicking the 'New Issue' button.

https://github.com/IntelRealSense/librealsense/issues