Program skipping frames

[idata]

Hello,

I have rosbag files that I am trying to process and want to be able to process each frame without skipping any. Currently, when I call poll_for_frames or wait_for_frames I either get a duplicate frame, skip a 1+ frames, or get the next frame. How can I make sure that I am getting every single frame, even if my program is taking more than 1/60th of a second to process each frame?

I am using C++. My current code is below:

rs2::config cfg;

cfg.enable_device_from_file("/path/to/file.bag", true);

rs2::pipeline pipe;

pipe.start(cfg);

while (true) {

rs2::frameset frameset;

pipe.poll_for_frames(&frameset);

std::cout << "Processing frame # " << frameset.get_frame_number() << std::endl;

}

I got this example from here: https://github.com/IntelRealSense/librealsense/blob/master/src/media/readme.md# using-rs2config-with-rs2pipeline librealsense/readme.md at master · IntelRealSense/librealsense · GitHub

Thanks!

[MartyG]

If a "For loop" is used then it forces the frame to be checked repeatedly until a maximum number is reached. For example, you might want it to check every frame for the next 30 frames before the loop can be concluded.

Here's a randomly chosen example of a For loop.

for (int i = 0; i < frameset.size(); i++) { rs2::frame new_frame = frameset[i]; int stream_id = new_frame.get_profile().unique_id(); view.second.frames_per_stream[stream_id] = view.second.colorize_frame(new_frame); //update view port with the new stream } } } }

The above code was sourced from the Multicam tutorial.

https://github.com/IntelRealSense/librealsense/blob/master/examples/multicam/readme.md librealsense/readme.md at master · IntelRealSense/librealsense · GitHub

[idata]

Hello MartyG,

Thanks for taking the time to help me out. My current understanding (and let me know if this is wrong) is that a frameset contains a view of each 'camera' in the device at a specific time. This way you can combine depth data and RGB data.

My problem is that I can not get consecutive framesets from the device (in this case the device is the recorded .bag file). The .bag file contains 10 minutes of 60FPS video, and I want to be able to get all 10 minutes * 60 seconds * 60 frames. Currently, my code skips certain frame numbers and repeats frames sometimes. My goal is to be able to fetch a specific frame one at a time when my program is ready.

The first solution I thought of was pausing the playback, and then force seeking it 1/60th of a second and getting a frame, but I don't think this worked.

I was hoping there was a built in way in order to get one frame at a time without skipping any. I understand in most situations when you are processing a live feed from a camera, you would want to skip frames you missed and get the most recent, but in this scenario it doesn't matter how long it takes to process the .bag file.

Thanks so much,

Tony

[MartyG]

I am not the best person to ask how framesets work, as RealSense stream programming is not my best topic. . I did though locate a discussion where someone wanted to store everything for future processing. Apparently the Librealsense team created a function called Keep() for this purpose that can be used with framesets.

https://github.com/IntelRealSense/librealsense/issues/1000 Saving frames for future processing · Issue # 1000 · IntelRealSense/librealsense · GitHub

https://github.com/IntelRealSense/librealsense/pull/1015 Adding frame.keep() API by dorodnic · Pull Request # 1015 · IntelRealSense/librealsense · GitHub

[idata]

I have seen that thread but I don't think that solution will work for me. These .bag files I have are 20GB+ and looping though the entire thing and calling .keep() on each frameset would use (if not more) 20GB of memory. I need to find a way to slowly step though the .bag files.

Hopefully someone who has experienced this issue will see this thread and be able to help me out, I highly doubt I am the first person in this use case.

Thanks for the help!

Tony

[MartyG]

In the previous SDK release 2.12.0 a tool called Convert to convert bag files to a probably smaller format was released.

https://github.com/IntelRealSense/librealsense/tree/development/tools/convert librealsense/tools/convert at development · IntelRealSense/librealsense · GitHub

[WAlea]

idata

were you able to find a solution for your problem?