Color stream capped at 15 FPS @ 1920x1080 (D415)

[THark]

Hi there,

During a 1920x180 color stream the RealSense viewer is only showing max 15 FPS, whilst 30 FPS is selected (D415, firmware 05.09.13.00).

The 1280x720 stream is correctly streaming at 30 FPS.

The camera is connected to a USB 3.2 port with the original usb cable and the used CPU is an i7-7700HQ.

I've tested multiple color formats, but I mainly focused on the YUYV format.

I saw that there were some CPU issues with the viewer that are being worked on, and I'm not sure if this is the bottleneck.

https://github.com/IntelRealSense/librealsense/pull/1844 Streaming cpu high usage fix by abernste · Pull Request # 1844 · IntelRealSense/librealsense · GitHub

https://github.com/IntelRealSense/librealsense/issues/744 [realsense2] high CPU utilization · Issue # 744 · IntelRealSense/librealsense · GitHub

So I've tried compiling the master code with OpenMP enabled and disabled, but both show max 15 FPS at 1080p for the color recording.

I've also tested a very simple script for saving 300 frames to a rosbag (see code snippet below) without any GUI being active.

This recording was also capped at 15 FPS (the recording takes 20 seconds instead of 10 s).

To minimize possible CPU overhead I've tried disabling the LZ4 compression but this didn't help either, the recording was still max 15 FPS.

The SSD should support writing 1080p @ 30 FPS (when compression was disabled recording @15 FPS was no problem).

So I'm wondering what could be the issue/ bottleneck that the camera only records/ streams at 15 FPS (and also not a little bit higher like 20 or 25 FPS).

Kind regards

int main(int argc, char * argv[])try

{

//Create a context and get the first device

rs2::context ctx;

auto devices = ctx.query_devices();

if (devices.size() > 0)

{

//Create a rs2::recorder from the first device, and desired file name

//'.bag' is the common extension for rosbag files

rs2::recorder device("my_file_name.bag", devices[0]);

//recorder "is a" device, so just use it like any other device now

}

rs2::pipeline pipe;

std::cout << "Start recording" << std::endl;

rs2::config cfg;

cfg.enable_record_to_file("output.bag");

//Create a configuration for configuring the pipeline with a non default profile

cfg.enable_stream(RS2_STREAM_COLOR, 1920, 1080, RS2_FORMAT_YUYV, 30);

pipe.start(cfg); //File will be opened in write mode at this point

for (int i = 0; i < 300; i++)

{

auto frames = pipe.wait_for_frames();

}

pipe.stop(); //File will be closed at this point

return EXIT_SUCCESS;

}

catch (const rs2::error & e)

{

std::cout << "RealSense error calling " << e.get_failed_function() << "(" << e.get_failed_args() << "):\n " << e.what() << std::endl;

return EXIT_FAILURE;

}

catch (const std::exception& e)

{

std::cerr << e.what() << std::endl;

return EXIT_FAILURE;

}

[MartyG]

Do you mean format YUY2?

The CPU load issues seem to have been mostly fixed in SDK releases 2.12.0 and 2.13.0. In 2.13.0, "the OpenMP compile flag is disabled by default, which can reduce the CPU utilization".

In the previous generation of RealSense cameras, there were a few rare cases where 30 FPS was being dragged down to 17 FPS because the lighting in the room that the camera was being used was too dim. Is the lighting in the location where your camera is being used at a relatively normal level?

[THark]

Yeah sorry, I meant the YUYV format and the lighting in the room is indeed normal.

I've just tested the same code on a different laptop (CPU: i7-6600U) in the same room which does record @30 FPS.

So I'm not sure why the previous laptop can't record @30 FPS.

[MartyG]

I just ran some tests in the RealSense Viewer with a D415 camera using the same settings (1920x1080, 30 FPS, YUY). The stream ran fine at around 30 FPS if only the RGB Camera was streaming. If I then enabled the Stereo Module stream too and had both RGB Camera and Stereo Module streaming, then the RGB stream speed alternated between 15 and 30 FPS.

Could you check please that you only have the RGB Camera stream active?

[THark]

Yes, I can confirm only the RGB stream is active/ recording.

[MartyG]

In the name of the camera at the top of the RealSense Viewer settings, does it say 'Intel RealSense D415' or does it say 'RealSense USB2'?

[THark]

Intel RealSense D415, and next to it the USB logo with version 3.2 behind it.

[MartyG]

Ok, thanks. We are narrowing it down now. Does the laptop that runs at 30 FPS have a better graphics chip than the one where it's bottlenecked at 15 FPS?

[THark]

There is a difference between the GPUs of the two laptops.

The laptop streaming at 30 FPS has an Intel HD Graphics 520.

The laptop running at 15 FPS has an Intel HD Graphics 630 and a Nvidia GeForce GTX 1050.

I've tried running the viewer through both GPU's from the latter laptop, but still no change in FPS.

[MartyG]

My main suspect of a cause is the USB port. Not all USB ports are made equal and you can get different stability from different ports, even on the same PC. Do you have another USB 3 port on your laptop that you can try the camera in?

[THark]

Yes, I indeed have tried a second USB port on the laptop but that didn't help either.

The 15 FPS laptop also has a USB C port, and I tried connecting the RealSense through an adapter (which also has a LAN port, VGA port etc) but it only shows up as a 3.0 connection.

When trying the 1080p RGB stream I get an error it took to long to catch a frame. So that's probably a problem with the adapter itself.

In the meantime I tried a third laptop which also streams with 30 FPS. So it could very well be the USB port of the 15 FPS laptop.

I'll see if I can find a USB C to USB C cable. Maybe that will work.

[MartyG]

There have been reported problems with using the 400 Series cameras with micro-USB 3 ports like the OTG format on mobile equipment such as the Up Board and Intel Compute Stick, apparently because they cannot supply the camera with the power that a full USB 3 port does.

[THark]

Today my laptop got a Thunderbolt 3 firmware update and this resolved the issues concerning the 15 FPS cap @ 1920x1080.

After this firmware update I can run the color stream @1080p and the depth and IR stream @720p simultaneously at 30 FPS without any issues.

So it was indeed a problem with the USB port.

[MartyG]

Awesome, thanks so much for the update!