Items with no label
3335 Discussions

SR300 precision & latency for face landmark detection

YShi7
Beginner
1,941 Views

Based on SR300 & SDK 2016 R2, I developed an application which adjust some display property in real-time according to view-point of end user, i,e., change display property according to current eye position got from SR300 & SDK landmark detection module.

Since viewer would be very sensitive to those display property, we want to get high precision position value with low latency as possible. However, I didn't find those parameters in SR300 product DataSheet (Document: 334531-001), and I also search intel Forums & Community. I got below thread about precision but can't answer my question. https://software.intel.com/en-us/forums/realsense/topic/675499 https://software.intel.com/en-us/forums/realsense/topic/675499

I also dumped the real data of my eye position when I move my head from right to left (x value goes increases). It shows that value has noise obviously. And I have to use Kalman filter to improve the result. below figure is the comparison of original data & after Kalman-filtered, the left sub-figure is the original one. I only plot x (horizontal) & z (depth) value.

I think the random of left sub-figure arise from the measure error of face landmark.

Another important thing is the latency, I didn't find answer to discuss what is the max or how to calculate the latency for face landmark detected. I think it would be helpful to estimate the more exact value of current eye position if we can get the real latency.

There is another thread in community to talk about such latency of hand detection while also can't answer my question.

All the best.

yshi

0 Kudos
3 Replies
MartyG
Honored Contributor III
228 Views

It is probable that if there was further data available about latency and precision for the SR300 camera then it would have emerged by now. The information that is available now is likely the limit of what can be known.

There is a new RealSense SDK called 'SDK 2.0' becoming available when the new super-powerful D415 and D435 cameras launch this month and is said by Intel to be SR300-compatible too. It is cross-platform but no specific information on OS support for it is available yet. It is possible that it may require Linux to operate rather than Windows though. If Linux is an option for you, there may be performance gains in that new SDK though over the previous Windows SDKs.

https://software.intel.com/en-us/realsense/400-series Intel® RealSense™ Depth Cameras D400-Series | Intel® Software

0 Kudos
YShi7
Beginner
228 Views

Hi Marty,

Thanks for your information!

Regarding to calculating the latency for face landmark detected, I've been wanting to find a way to convert the time value got by QueryFrameTimestamp(), i.e., the time domain of RealSense I guess, to the time value got from Windows. The QueryFrameTimestamp() would be the time when the latest data (frame) sampled.

I think if we know how to convert to OS time, then we can know how fresh of the position value when the time we use the position value to control something. I also write such part code to get timestamp of RealSense sample & Windows system time like below. However I can't find way to convert one the the others. It would be appreciated if you can tell some tip.

class CSystemClock {

public:

double m_dobDff;

CSystemClock() {

LARGE_INTEGER largeInt;

QueryPerformanceFrequency(&largeInt);

m_dobDff = largeInt.QuadPart;

}

int64_t getCurUs() {

LARGE_INTEGER largeInt;

QueryPerformanceCounter(&largeInt);

int64_t cur = largeInt.QuadPart;

cur = cur * 1000 * 1000 / m_dobDff;

return cur;

}

};

PXCFaceData *m_pFaceData;

m_pFaceData->Update();

//got time stamp of the sample from RealSense

int64_t qryTime = m_pFaceData->QueryFrameTimestamp();

qryTime = qryTime / 10000; //QueryFrameTimestamp unit is 100ns, convert to ms.

Best.

yshi

0 Kudos
MartyG
Honored Contributor III
228 Views

Coincidentally, I was doing game programming with the system clock yesterday in the Unity game creation engine. I don't know if the equation I used will be of any use, but I'll share it anyway.

- So you take the 24 hour PC clock time,: there should be 86400 seconds in 24 hours.

- Do %60 on this value to get the number of minutes in a day.

86400 % 60 = 1440

- Then do %60 on the minutes value to get the hours of the day

1440 % 60 = 24

The process works in the opposite direction to convert hours into seconds too on the 24 hour clock, using multiplication instead of %.

Hours x 60 = minutes, and minutes x 60 = seconds

********

So, if a QueryFrameTimestamp() second is the same length as a PC clock second, maybe a conversion to 24 hour clock could be done that way?

In my game project, I used these equations to construct a timer in the 24 hour clock which counted up from 0, which was 0:00:00 (midnight). So when it was running, you could see how many seconds had passed in 24 hour clock time So if 3 seconds passed, the 24 hour clock time in 24 hours was 00:00:03. If you did a printout of the QueryFrameTimestamp() value in 24 hour time and the PC clock's seconds count in 24 hour time, you could see how closely the two were aligned, perhaps.

My apologies if this thinking isn't of any use to you!

0 Kudos
Reply