Rounding of depth value SR300 to 1 mm (SDK 2016 R2)

[THark]

Hi there,

Currently I'm trying to read the real world coordinates of a pre recorded ".rssdk" file hence the use of the old SDK.

However, the depth values I'm getting keep being rounded to 1 mm.

I've seen multiple topics about this issue and I've tried implementing the recommended solutions.

https://software.intel.com/en-us/forums/realsense/topic/606914 Getting precise depth data without QueryVertices()

https://software.intel.com/en-us/forums/realsense/topic/597722 Depth Resolution of Real Sense Camera

However the implementation of the suggestions were not successful. So I'm not sure if my own implementation is wrong or that my pre recorded rssdk file does not contain depth values with a higher accuracy.

I've tried 3 different implementations in C++ shown below here (with a few checks removed).

A typical output I would get from this script are the following values:

pos3D[i].x = 0.305 mm

pos3D[i].y = 145.285 mm

pos3D[i].z = 520.000 mm

dPixels[i] = 520.000 mm

dPixelsRaw[i] = 4160

zconvert = 520.000 mm

int wmain()

{

// Filename

pxcCHAR file[1024] = L"filename.rssdk";

// create the PXCSenseManager inluding the instance

PXCSenseManager *sm = 0;

sm = PXCSenseManager::CreateInstance();

// Set file recording or playback

sm->QueryCaptureManager()->SetFileName(file, false);

// Select the color stream

sm->EnableStream(PXCCapture::STREAM_TYPE_DEPTH, 640, 480);

// Initialize

sm->Init();

// Analyze first 10 frames

for (int i = 0; i < 10; i++) {

// This function blocks until a color sample is ready

if (sm->AcquireFrame(true) < PXC_STATUS_NO_ERROR) break;

// Retrieve the depth sample

PXCCapture::Sample *sample = sm->QuerySample();

PXCCapture::Device *device = sm->QueryCaptureManager()->QueryDevice();

pxcF32 depthUnit = device->QueryDepthUnit();

// Example 1: Read image data in F32

PXCImage::ImageData depthImage;

sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH_F32, &depthImage);

float *dPixels;

dPixels = (float*)depthImage.planes[0];

// Example 2: Projection with QueryVertices in 3D F32

PXCImage::ImageInfo imgInfo = sample->depth->QueryInfo();

int depth_width = imgInfo.width;

int depth_height = imgInfo.height;

int num_pixels = depth_width * depth_height;

PXCProjection * projection = device->CreateProjection();

PXCPoint3DF32 *pos3D = new PXCPoint3DF32[num_pixels];

sts = projection->QueryVertices(sample->depth, &pos3D[0]);

// Example 3: Raw Depth

PXCImage::ImageData depthImageRaw;

sample->depth->AcquireAccess(PXCImage::ACCESS_READ, PXCImage::PIXEL_FORMAT_DEPTH_RAW, &depthImageRaw);

pxcU16 *dPixelsRaw;

dPixelsRaw = (pxcU16*)depthImageRaw.planes[0];

// Display values

for (int i = 0; i < num_pixels; i++)

{

// Only output values to command prompt

if (pos3D[i].x > 0)

{

// Convert the raw value to world coordinates

float zconvert = float(dPixelsRaw[i])*(depthUnit / 1000.0f);

cout << fixed << setprecision(3) << pos3D[i].x << " " << pos3D[i].y << " " << pos3D[i].z << " " << dPixels[i] << " " << dPixelsRaw[i] << " " << zconvert << endl;

}

}

// Go fetching the next sample

sm->ReleaseFrame();

}

sm->Release();

}

[MartyG]

I'll link into this discussion jb455 the RealSense stream programming expert from the link you saw, 'Getting float depth data'. Hopefully he can provide useful suggestions for your problem.

[jb455]

The only thing I can think of is if the float type you're using is 16-bit instead of 32 and it's truncating the values. But I think the C++ float type is 32 bit so you're probably alright there. Everything else looks OK.

It's possible the rssdk file doesn't store the full precision, either due to a bug or by design if they wanted to save space - hopefully someone from Intel will be able to confirm. In the meantime, you could try decoding it yourself - https://software.intel.com/en-us/blogs/2015/10/15/how-to-record-and-playback-streaming-sequences-in-intel-realsense-sdk this may help - to see if the issue is with recording (not storing full precision) or playback (not reading the file properly).

BTW, the new SDK can record and play back colour/depth streams from the cameras, though I don't think it's compatible with rssdk files (and unlikely to be https://github.com/IntelRealSense/librealsense/issues/144 by the looks of things) so if you need to work on these specific files and new ones can't be recorded with the new SDK you may be SOL.

[THark]

Great, thanks for the reply! I will look into it and report my findings.

The C++ float is indeed 32-bit but for clarity sake I changed the floats to pxcF32 types in my code. The results were not affected by this change.

A part of my recordings can't be recorded again, however I'm still creating some new recordings so if the depth accuracy indeed is being rounded in rssdk files I'll switch to the new SDK.

Unfortunately, I'm also using the 3D face/landmark detection algorithms from the old SDK, and as far as I know these are currently not available in SDK 2.0.

I know it's possible with OpenCV in 2D and then performing a projection to 3D ( ) but it's a shame there is no native implementation/correction for the 3D data as with the old SDK.

[THark]

During a live stream, the code from above indeed returns float values for the depth data as expected. Therefore, it indeed seems to be a recording or playback problem of the rssdk file itself. My first try was to decompress the original file with the RSSDK ClipEditor in case it might help (I didn't expect it would, but who knows), but this had no effect on the precision of the depth values. Decoding the rsssdk file myself might turn out to be a bit too difficult, so I'm first trying to see what happens if I disable the LZO compression for the depth data, to see if I can increase the accuracy for a new recording. At the moment the registry edit to disable the compression didn't work properly (https://software.intel.com/en-us/blogs/2015/10/15/how-to-record-and-playback-streaming-sequences-in-intel-realsense-sdk https://software.intel.com/en-us/blogs/2015/10/15/how-to-record-and-playback-streaming-sequences-in-intel-realsense-sdk) but hopefully I can fix this soon and see if this helps.

Edit - After disabling the LZO compression the depth data is still being stored as rounded numbers. So, as far as I can tell it is not possible to store the float depth values in a rssdk file (or read them properly again).