Compression Formats for RGB-D Streaming

[PSnip]

Hi,

(1) We want to process the RGB-D stream from realsense camera on the cloud. Can some one suggest what kind of compression could be used on RGB-D data prior to streaming? Are there any practical RGB-D compression formats? I could not find much information on this on the web.

(2) Are there any open-source solutions for this? Or is there any company which offers commercial solution for this?

(3) One option I see is to compress & stream the RGB & D data separately. Even if we were to use this method, I have some questions

(i ) What kind of compression formats could be used on D data? I could find very few reference on Depth data compression. Those too mainly some research papers.

(ii ) How can we preserve the frame synchronization between RGB & D streams if these were to be streamed separately?

Regards,

PaintedSnipe

[MartyG]

I could not find an official compression format, but rather different approaches to compression that engineers have tried, found by googling for 'rgb-d compression format'. Of the small amount I saw, a system called CAR seemed like the most accessible and well-documented. I culd not tell whether it is commercial or open source, though the effort to guide people in setting up their own system suggests open-source. You may wish to inquire to them further about that.

https://team.inria.fr/stars/software/car-complex-activity-recognition-component-installation/ CAR (Complex Activity Recognition) module – Stars

CAR - Recording compressed RGB-D data (section 2 of the page)

https://team.inria.fr/stars/software/car-complex-activity-recognition-component-installation/car-component-testing-car/ CAR – Testing the CAR component – Stars

CAR - Reading back the compressed data.

https://team.inria.fr/stars/compressed-rgbd-video-reader/ Compressed RGBD video reader – Stars

Other compression methods, which you may have seen during your own researches, include:

3D Image warping

http://ieeexplore.ieee.org/document/7067400/ http://ieeexplore.ieee.org/document/7067400/

Lossless and lossy streaming

http://ieeexplore.ieee.org/abstract/document/7017650/ http://ieeexplore.ieee.org/abstract/document/7017650/

Compressed sensing and sparse coding

https://graphics.tu-bs.de/upload/publications/foerster2015pcs.pdf https://graphics.tu-bs.de/upload/publications/foerster2015pcs.pdf

Compressing RGD-D data for cloud robotics with the open source LibAV tool

https://github.com/rapyuta/rapyuta-kinfu/wiki Home · rapyuta/rapyuta-kinfu Wiki · GitHub

Regarding the question of synching, the new SDK 2.0 ca use the D415 and D435 cameras to provide advanced stream synchronization. These functions may perhaps be of use to you.

https://github.com/IntelRealSense/librealsense/blob/development/doc/rs400/external_devices.md librealsense/external_devices.md at development · IntelRealSense/librealsense · GitHub

[PSnip]

Thanks Marty,

I will explore this options. You are so prompt in helping everyone on this Forum. Thanks again.

Regards,

PS

[MartyG]

You're very welcome. The more that the community succeeds, the higher its ambitions can reach. Success breeds success