3D facial scanning

[NMant1]

Hello there,

I'm using the SR300 to make a 3D facial scan, which will be used in an external algorithm to identify the face. This is done in VS in C# . Until now, I create the point cloud with the following specifications:

StreamType: STREAM_TYPE_DEPTH

PXCMImage.PixelFormat: PIXEL_FORMAT_DEPTH_RAW

Then I apply the QueryVertices function, which converts the depth data to vertices.

I've tried to improve the quality in some ways, like using an average of 5 frames, but until now, the results have been insufficient for the identifying algorithm. Now I've looked through the datasheet, and have found some filters and presets which might be able to help, like the preset "PRESET_FACE_LOGIN", and the filters"FILTER_SCANNING" and "FILTER_SKELETON".

Are there good ways to increase the fidelity of the facial image? Do these presets and filters help, and how do I apply them?

Thanks a lot for reading,

Nahuel

[MartyG]

There was a doc accompanying the sample that also had very detailed code notes. I've attached it to this message as a PDF for your easy access.

[NMant1]

With some work, and use of the code notes I was able to get the stream working. I can now make images of sufficient quality using the auto laser as jb455 suggested. I tried to work with some ND filters to weaken the light coming into the camera. Although this reduced the vertical lines a little in the front of the face, the light was no longer strong enough and parts further away from the camera started to disappear from the model.

Thanks a lot for your help!

[MartyG]

You're very welcome! I'm glad JB's approach was able to help you. Please come back to the forum any time you need help

[PKuma35]

NahuelM MartyG

I am facing a similar problem. Though I am able to get better data by having the subject closer to the camera, I cannot remove those vertical lines easily.

And I don't think the Intel Stitching produces a very accurate geometry.

How did you come over this problem? Can you please explain?

Please find my current single frame attached for your reference.

[MartyG]

I think you are in a difficult situation where you need to get close to the skin to get better data, but the close proximity to the skin is causing the surface of the scan to break up, like in the scans of skin highlighted in earlier discussions.. Lines in scans have been an issue with RealSense ever since it debuted in 2014. You can only minimize it but not totally eliminate it.

Lighting conditions in the location you are taking the scan in can affect how much noise is generated, as if the room is too dark or too bright then an IR Emitter component inside the camera that helps the IR sensor with exposure can cause the IR sensor to become saturated. You can turn the IR emitter off with scripting.

Librealsense

RS_OPTION_R200_EMITTER_ENABLED

Example script:

* Enables / disables R200 emitter

*/

public void setEmitterEnabled(int value) {

setOption(RealSense.RS_OPTION_R200_EMITTER_ENABLED, value);

}

RealSense SDK

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?setdsenableemitter_device_pxccapture.html Intel® RealSense™ SDK 2016 R2 Documentation

[NMant1]

Hey Praveen,

I don't use any form of stitching at the moment. Just a single frame. It seems to me you are using the raw stream, instead of filtering it with the software. Try using the SetIVCAMFilterOption() function and setting it to 2 or 3 to get a better result. The thing that greatly reduced the lines for me is setting the SetIVCAMLaserPower() function to 1 (auto) as jb455 suggested, which means the projector power will lower when the subject is closer, and stop from overexposing the camera image. Also, I think you are way too close to the camera to get a decent image. You will have to move back a little even with the above changes. Furthermore, the quality of the depth is greatly reduced when close to outside light, so stay as far as you can from any window. I managed to get the following result:

You can see that the ridges are far from gone, but they are small enough for me to be acceptable.

I hope this helps, good luck!

[PKuma35]

Thanks for your quick reply NahuelM

Are you using the Intel SDK or the LibRealSense?

[NMant1]

The RealSense SDK (https://software.intel.com/en-us/intel-realsense-sdk/download Intel® RealSense™ SDK | Intel® Software). I'm using C# in Microsoft Visual Studio (community edition). I haven't worked with LibRealSense, so I can't help you there, I'm afraid.

[PKuma35]

NahuelM

Great. I am using the same Intel Real Sense SDK as well in C# .

So you extracted single frames with tweaking only the above-said settings, is that correct?

[NMant1]

Yes, that's all I changed. I'm using a SR300, by the way. I'm assuming you are as well.