What is accuracy of full hand tracking using SDK 2016 R3 documentation.

[KPark22]

I want to do full hand (skeletal) tracking using 2016 R3 SDK (old SDK). Does anyone know the accuracy?

[MartyG]

I worked for a couple of years with hand tracking on the 2016 SDK and found it to be very good. It had some problems with two handed use, as if one hand moves in front of the other then tracking of the hand behind may be lost. I used to be sure to move my hands in a rhythm that alternated the hand positions between front and back so that tracking of both left and right hand remained active.

​Another issue with complex use I experienced was that if using both palm tracking and finger joint tracking at the same time, moving the palm would cause the finger joints to change as well.

I ended up building a lot of custom solutions. These are detailed in a large range of step by step guides for the 2016 SDK at the link below.

https://software.intel.com/en-us/forums/realsense/topic/676139

[KPark22]

Thanks for the answer. And your step by step guides are great!

That would be really helpful.

Have you seen any document or research that provides numerical finger tracking accuracy result of the SDK?

[MartyG]

I'm glad you liked the guides. :)

I cannot recall seeing official numeric guidance about the tracking accuracy. I do recall that it supported being able to set the sensitivity of hand opening and closing, so that hand open or close state would be detected when the fingers were closed at a certain percentage range specified by the user

Although joint tracking is very good, gesture detection (thumb up / down, hand open / close, swipe, wave, pinch etc) had more difficulty being detected in the 2016 SDK. My own experience was that the most successful hand gestures to detect were hand open / close, thumb-index pinch, and thumb-up. Detection of other gestures such as wave and swipe were less reliable.

In regard to which finger joints were the best to use, I found that the middle joint (the knuckle in the middle of the finger length) provided the best results when the hand was both open and closed. This is because these joints remain facing-forward to the camera when the hand is closed into a fist, whereas the fingertips may be hidden from the camera's view when the hand is closed.

It is worth adding that the 2016 SDK can sort-of do leg tracking too. This is because it detects large flat-ish areas like the knee and foot soles as a palm, and the toe joints as finger joints. The camera's joint tracking cannot see through clothes though, so it requires bare legs and feet to work.

I did some tech-demo videos of joint tracking with the 2016 SDK a few years ago that may be of interest to you.

https://www.youtube.com/watch?v=T_V8SNwLTrY

https://www.youtube.com/watch?v=md8HuKREfXE

[KPark22]

Thank you!!