Items with no label
3335 Discussions

Realsense Viewer Recording Framerate

APu
New Contributor I
4,081 Views

I am using D435 camera to take videos at 1280x720 resolution at 30fps. Due to frame dropping issue of pipeline, I used MATLAB's toolbox to iterate through the .bag file. However, I see that I only read 3720 frames in 356 seconds, which is approximately 10-11 fps. Has anybody else run into this issue? Or is there something I am doing wrong?

0 Kudos
15 Replies
MartyG
Honored Contributor III
1,902 Views

The user Leponzo had some suggestions recently about getting all the frames in a bag file and using them with MATLAB.

0 Kudos
APu
New Contributor I
1,902 Views

Thanks for linking me. I've seen the discussion before which is why I decided to use MATLAB for post-processing, but it seems as if the software itself is not taking frames at the rate that I would like it to be (30fps). I'm not sure why this is the case.

0 Kudos
MartyG
Honored Contributor III
1,902 Views

Are you reading from a pre-recorded bag file or are you post-processing the data as it arrives from the camera in real-time, please?

0 Kudos
APu
New Contributor I
1,902 Views

HI MartyG

Thanks so much for your help. I am reading a pre-recorded bag file on MATLAB. When I look at the properties of the .bag file, I see that there are approximately 3720 frames taken in the duration of about 356 second.

I recorded this bag file with Intel's Realsense Viewer Tool (ver 2.13.0) on Windows 10, and am on the latest version of firmware for the cameras.

0 Kudos
MartyG
Honored Contributor III
1,902 Views

A new tool called bag2matlab was released in June that claims good performance and can be launched from within MATLAB. It directly reads the bag data into the MATLAB workspace.

https://nimbus.unl.edu/2016/06/exporting-ros-bag-files-to-matlab-v2-0/ Exporting ROS Bag Files to Matlab, v2.0 | NIMBUS Lab

0 Kudos
APu
New Contributor I
1,902 Views

The problem isn't importing the bag file to MATLAB. I can do that just fine. I'm confused as why the duration and number of frames I have do not match 30fps.

0 Kudos
MartyG
Honored Contributor III
1,902 Views

Bag files can be very large, so I wonder if MATLAB is struggling to cope with the file size. There is some official advice here on the subject:

https://uk.mathworks.com/help/matlab/large-files-and-big-data.html Large Files and Big Data- MATLAB & Simulink- MathWorks United Kingdom

0 Kudos
APu
New Contributor I
1,902 Views

When I view the bag file on the Realsense Viewer afterwards, the video looks the same as my video exported by MATLAB. There are instances that i can visibly see that the video is not 30 fps as it should be, so I don't think it's exclusively a problem with MATLAB, but I can check it out as well.

0 Kudos
APu
New Contributor I
1,902 Views

I just experimented and when I only record a depth stream or a RGB stream, the framerate is flawless at 30fps. Is there any reason why streaming both at the same time degrades the framerate?

0 Kudos
MartyG
Honored Contributor III
1,902 Views

What are the capabilities of your PC hardware like? On my slow laptop with integrated Intel HD graphics, I get smooth FPS with depth and IR together but lose speed when RGB is activated too.

0 Kudos
APu
New Contributor I
1,902 Views

Hi MartyG

My processor and its specs can be found here: https://ark.intel.com/products/92980/Intel-Xeon-Processor-E5-2623-v4-10M-Cache-2_60-GHz Intel® Xeon® Processor E5-2623 v4 (10M Cache, 2.60 GHz) Product Specifications

My computer doesn't seem slow but it's somehow affected when recording both depth and IR together. Is there a recommended hardware spec that should be utilized to take both streams simultaneously?

0 Kudos
MartyG
Honored Contributor III
1,902 Views

Though your Xeon processor is ageing, its strong enterprise specification means it can still likely give good performance compared to newer consumer processors. Graphics technology does also have an impact on performance though when a heavier visual demand is placed on a machine.

RealSense 400 Series cameras do not have a strict recommendation for specification, other than an Intel or ARM processor and a preference for USB 3.

0 Kudos
APu
New Contributor I
1,902 Views

MartyG

My laptop is an Intel Core i7 and it doesn't even record 1280x720 at 30fps without slowing down.

Are you aware of anybody else who had this problem and what they did to amend it? If there is no strict recommendation for specifications, I would expect for most machines to be able to stream video at the resolution and framerate that the camera promises.

0 Kudos
MartyG
Honored Contributor III
1,902 Views

The 400 Series is intended to be usable on many types of hardware and software platforms. This necessitates that the minimum spec be extremely flexible. This is one of the reasons for an open source SDK that encourages compiling the SDK for your specific hardware instead of trying to provide a one size fits all solution that may not run on more exotic systems.

Having said that, the Vision Processor D4 component of the 400 Series camera is designed to do high resolution calculations without the need for a dedicated GPU, thus reducing the load on the processor. If there is slowdown, it may be more to do with the USB port than the machine's graphics capability. For example, using the camera at high resolution or FPS on some devices may cause errors as the USB port experiences 'overflow', and those errors reduce or disappear when resolution / FPS is lowered. This is why the faster USB 3.1 standard is becoming increasingly essential.

0 Kudos
ASriv7
New Contributor I
1,902 Views

Yes. Also, the frame numbers reported by MATLAB and the Viewer match (e.g. 698, 699, 700, 701, 703, . . . it dropped 702 in BOTH MATLAB and the Viewer), so MATLAB is not the issue here (in fact, as mentioned in an earlier post, it was the only tool I found that reliably reads all the data from a bag file for Intel D400 series).

0 Kudos
Reply