Items with no label
3335 Discussions

SR300 - crash while facetracking multiple people

ASere5
Beginner
1,514 Views

Hi everyone,

I'm trying to make a face tracking app for my Bachelor degree.

The app will run in an artistic gallery, which means running 16 hours straight, with people experiencing the installation at anytime during these 16 hours, without supervision of any kind. Any crash of the app will completely disrupt the experience of the user, so we can't afford this.

Thing is, if you read the title you already know that the app is still crashing at the moment.

So here's a description of what I need, the conditions in which the bug happens, the conditions in which the bug seems not to trigger, the info I could get on that bug and what I tried / will try to work around it.

Note : I'm working with a SR300, the essentials and face SDKs, and I'm using C++ with Qt on Windows 10.

What I need

I need an app that can retrieve the following features of one's face :

- position x and y

- width and height

In short, the bounding box of the face. Pretty basic stuff, here's an example to illustrate this :

One and only one person should be tracked at anytime, meaning that even though the app must support multiple people in the camera's FOV, I only need one face bounding box at a time.

And here's the problem : multiple people can be in the FOV at the same time, they'll be detected, but I always track the first face in the list of all faces detected. But sometimes, the app crashes.

Conditions in which the bug happens

First things first, we need to make the difference between detected and tracked.

The camera can detect multiple faces (meaning recognize "oh hey see there are 3 faces in here") and track only one (meaning retrieve the infos of position and size of only one face).

Here are the conditions in which I could trigger the bug every time.

When 3 faces are detected, if the tracked one leaves the FOV, the app crashes. Here's an illustration :

Here you can see the video stream at the time of the crash, right when I leave the FOV of the camera. I could simulate the multiple people with this image of me and my girlfriend (only thing I had, but it worked alright so don't judge ).

As you can see on the right-hand side of the image, "Nb tracked people" equals 3. This should be named "Nb detected people" to be consistent with the definition above btw. This is directly the value returned by SomeFaceData->QueryNumberOfDetectedFaces() at the moment of the crash.

This only happens if it's the currently tracked face that leaves the FOV, but any of the detected but untracked faces cane leav the FOV as they wish, it does not trigger a crash.

I could recreate this crash (or at least make crash, maybe for a totally unrelated reason) with the C++ face tracking exmaple, found in the samples of the SDK.

Conditions in which the bug seems not to trigger

This same bug has been triggered with only 2 people detected (and maybe even one), but it's rare. I could not find a setup to trigger the bug consistently without at least 3 faces detected.

This bug doesn't seem to trigger more often when doing kissing-like gestures. I'm saying this because I read this thread : but it doesn't seem to be related. Can't be sure though.

Infos I could get on the bug

Here are all the infos I could gather on this bug.

First of all, the screen I get in QtCreator when the app is launched in Debug mode :

Coming with its full cryptic stacktrace :

We can't see it clearly in here, but every single one of these calls appears in assembly only when selected. There is no decently readable code to go to and say "There's a pointer access that you didn't check for nullptr !", unless you debug ASM code for breakfast, in which case I welcome your input on the matter.

Finally, my bug was strangely close to this one : , even though I'm not using Unity at all. I can't tell if they're related at all though.

What I tried / will try to work around the bug

I tried to limit the number of detected faces to one.

I made sure to only track one face in the config of the camera like this :

pxcI32 nbFaces(1);

m_config->detection.isEnabled = true;

m_config->detection.maxTrackedFaces = nbFaces;

//...

m_config->ApplyChanges();

I thought that this would limit the number of detected faces, but instead limits the number of faces I can track (apparently, this is not stated in the documentation, it's just my personnal observation).

Later in my code, I verify it like this :

int nbFaces = SomeFaceData->QueryNumberOfDetectedFaces(); // nbFaces = 0, 1, 2 or 3...

FaceData::Face* trackedFace = SomeFaceData->QueryFaceByIndex(0); // Always take the first face in the list.

If I did :

FaceData::Face* trackedFace = SomeFaceData->QueryFaceByIndex(1);

for example, this would trigger an exception when trying to access trackedFace, or its proper...

0 Kudos
6 Replies
MartyG
Honored Contributor III
284 Views

You are the third person this week to have reported the issue with multiple faces, and a crash occurring when one of the faces enters and then leaves tracking range. I have been connecting affected developers together so they can share their experiences, and also provide evidence that may help Intel to track down the cause of the issue.

0 Kudos
ASere5
Beginner
284 Views

Hi Marty, thanks for your quick response.

I did see some really recent threads reporting similar crashes.

I tried to be as complete and specific as possible in this thread, hoping it could help both user and developers of the SDK in resolving these matters.

Thanks again.

0 Kudos
MartyG
Honored Contributor III
284 Views

Thanks too for providing details of your project. I saw another user recently who also wanted to measure the face.

0 Kudos
JCele
Beginner
284 Views

Hi,

you said "When 3 faces are detected, if the tracked one leaves the FOV, the app crashes."

I think that there may be a chance that the problem lies here for me too : in the screenshot in my issue, the screenshot of the sdk app that crashed has ALERT_FACE_OUT_OF_FOV in its status bar.

Sadly my code was already tracking a single face (and thus always accessed face 0), yet I got the crashes.

@Intel: maybe if we got the pdb of libpxcface_idc.dll we could give you more meaningful stack traces ?

0 Kudos
ASere5
Beginner
284 Views

Hi,

I didn't get the ALERT_FACE_OUT_OF_FOV, my app just plain crashes without any signal or note, but that might be just this too.

And tracking only one face doesn't sove the problem for the camera apparently, so that's coherent between us too.

I agree with you for the pdb files. The symbols for the libpxcface_idc and libpxccore mwill get us some better stacktraces indeed.

Maybe they're on the github ? https://github.com/IntelRealSense/librealsense GitHub - IntelRealSense/librealsense: Cross-platform API for Intel® RealSense™ devices

I never worked much with pdb files, I don't know if that's something that's usually committed or not.

Cheers

0 Kudos
idata
Employee
284 Views

Hi guys,

 

 

I would suggest you to take a look at this link, regarding an issue with the libpxcface_idc.dll library.

 

/community/tech/realsense/blog/2017/06/20/known-issue-crash-in-libpxcfaceidcdll-in-realsense-sdk-r3 https://communities.intel.com/community/tech/realsense/blog/2017/06/20/known-issue-crash-in-libpxcfaceidcdll-in-realsense-sdk-r3.

 

 

Regards,

 

Pablo M.
0 Kudos
Reply