Items with no label
3335 Discussions

Best environment for skeleton tracking?

PKlok
Beginner
2,337 Views

Hi,

 

We plan to show our skeleton tracking solution made with Intel realsense and Nuitrack on an event coming up shortly. However, we have a question we cannot test ourselves right now.

 

At the office we can test the application, but we currently do not have the stand setup we will use on the event(lighting, floor etc.). But obviously we would like to prepare as much as possible and make sure our Realsense D415 can track the skeleton properly. 

 

So my question is, is there any information on what can cause the camera to be less accurate(environment wise). We noticed at times it reads the skeleton a bit better/worse. Anything to do with lighting, ground, clothes etc? Anything that we can prepare for would be highly appreciated. What is the best setup overall to make sure everything gets tracked properly?

 

NOTE: Below are some images of the panels we will have on the stand.

 

1. Fiberboard with darkblue hard pressure laminate (hpl) top

1.jpg

2.Fiberboard with white hard pressure laminate (hpl) top

2.jpg

3.Polished Multiplex (pinewood structure)

3.jpg

 

Thanks in advance,

 

Greetz,

 

Patrick

0 Kudos
4 Replies
MartyG
Honored Contributor III
251 Views

Here is some advice for you, Patrick.

 

  • Avoid fluorescent lighting such as strip-lights if possible and use bulbs instead. This is because the heated gas inside fluorescent lights can flicker at a frequency hard to see with the human eye and cause depth noise. If there are going to be fluorescent lights at the stand then running the camera stream at an FPS close to the operating frequency of the lights may help. For some people it has been 30 FPS and for others it has been 60 FPS.

 

  • If your stand has a ceiling then you may be able to get more control over this by fitting your own choice of lights to the ceiling and blocking out the roof lighting at the event location.

 

  • If your solution involves using a projected dot pattern from the IR emitter, you may get less noise if an external LED projector is used instead of the internal laser-based IR emitter of the camera. An LED projector reduces 'speckle' on the image.

 

  • Avoid clothes with black or grey on them, as it can absorb the laser light and leave a black empty area on the image that is shaped like the dark area (such as a cable-shaped black area when scanning a black cable).

 

  • Polished surfaces may create disruption in the image, especially if a light-source is shining on them. Reflections can be dampened by putting anti-reflective film over their surface. Here is an example:

 

http://prodisplay.com/products/anti-reflective-anti-glare-film/

 

  • I saw a report from a user recently that said that they had image quality problems with a grey concrete surface. If the stand has a non-reflective floor surface that is preferably not black or grey then you should be okay.

 

  • Intel's body tracking demonstration stand at the Sundance Festival in January 2018 may give your some further clues about optimum stand aesthetics.

 

1.png

 

 

 

  • Having textured objects in the camera view can help the camera to lock onto the scene because it adds analyzable detail to the location. In Intel's demo stand, they seem to achieve this with a large star-field poster as a backdrop.
PKlok
Beginner
251 Views

Hi MartyG,

 

Thanks a lot for the tips! We will be looking into these in the upcoming week and we will see how much we can optimize.

 

Could you explain a bit more about tip 3? I'm not sure i quite get what you mean...

 

0 Kudos
MartyG
Honored Contributor III
251 Views

If you mean the LED projector ... RealSense cameras have a built-in IR Emitter component that can project a semi-random pattern of dots onto the objects in a room. This helps the camera to analyze the room, as it adds texture to the surfaces of objects that may otherwise be lacking surface detail (such as walls and doors).

 

A disadvantage of this laser-based system though is that it can create 'speckles' on the image. You can also purchase external projectors that also project a dot pattern but do so with LED light instead of laser light. This reduces the amount of speckle on the image. Another advantage of these external projectors is that they can be moved and shaken without disrupting the camera's depth analysis of the scene.

 

Intel's camera tuning guide has the following information about external projectors:

 

**********

 

a. While the internal projector is very good (shown below), it is really quite low power and is not designed to illuminate very large rooms. It is therefore sometimes beneficial to add one or more additional external projectors. The D4xx camera will benefit from any additional texture projected onto a scene that has no inherent texture, like a flat white wall. Ordinarily, no tuning needs to be done in the D4 VPU when you use any external projectors as long as it is on continuously or flickering at >50KHz, so as not to interfere with the rolling shutter or auto exposure properties of the sensor.

 

b. External projectors can be really good in fixed installations where low power consumption is not critical, like a static box measurement device. It is also good for illuminating objects that are far away, by adding illumination nearer to the object.

 

c. Regarding the specific projection pattern, we recommend somewhere between 5000 and 300,000 semi-random dots. We normally recommend patterns that are scale invariant. This means that if the separation between the projector and the depth sensor changes, then the depth sensor will still see texture across a large range of distances.

 

d. It is possible to use light in visible or near-infrared (ranging from ~400-1000nm), although we normally recommend 850nm as being a good compromise in being invisible to the human eye while still being visible to the IR sensor with about 15-40% reduction in sensitivity, depending on the sensor used.

 

e. When using a laser-based projector, it is extremely important to take all means necessary to reduce the laser-speckle or it will adversely affect the depth. Non-coherent light sources will lead to best results in terms of depth noise, but cannot normally generate as high contrast pattern at long range. Even when all speckle reduction methods are employed it is not uncommon to see a passive target or LED projector give >30% better depth than a laser-based projector.

 

********

 

The camera tuning guide is an excellent resource for optimizing camera performance in general.

 

https://realsense.intel.com/wp-content/uploads/sites/63/BKM-For-Tuning-D435-and-D415-Cameras-Webinar_Rev3.pdf?language=en_US

 

 

 

PKlok
Beginner
251 Views

Thanks Marty. The tips are really helpfull. We will report back!

0 Kudos
Reply