Items with no label
3335 Discussions

realsense SR300 R3 SDK+ Unity3d5.5.2f1 Bug!

天虞
Beginner
1,994 Views

Hi everyone,

This sample can be used in Editor, and can be released using exe.

C:\Program Files (x86)\Intel\RSSDK\sample\core\RawStreams.unity\ ========>(Only Core)

However, if you use Core+Face,

Can be used in the Editor, but the release of exe can not work.

-------------------------------------------------------------------------------------

c# script =>start()

faceModule = FaceModule.Activate(sm);

 

if (faceModule == null)

 

{

 

Debug.LogError("FaceModule Initialization Failed");

}

 

//faceModule.FrameProcessed += FaceModule_FrameProcessed;

FaceConfiguration moduleConfiguration = faceModule.CreateActiveConfiguration();

 

if (moduleConfiguration == null)

 

{

 

Debug.LogError("FaceConfiguration Initialization Failed");

}

 

moduleConfiguration.TrackingMode = TrackingModeType.FACE_MODE_COLOR;

 

moduleConfiguration.Strategy = TrackingStrategyType.STRATEGY_RIGHT_TO_LEFT;

 

moduleConfiguration.Detection.maxTrackedFaces = 1

 

moduleConfiguration.Landmarks.maxTrackedFaces = 1

moduleConfiguration.Detection.isEnabled = true;

 

moduleConfiguration.Landmarks.isEnabled = true;

moduleConfiguration.Pose.isEnabled = false;

...........

-------------------------------------------------------------------------------------

Log:

UnloadTime: 0.882964 ms

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/.\libpxcface_c

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/.\libpxcface_c

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/.\libpxcface_c

Fallback handler could not load library D:/Unity3d Project/RealSense Unity Project/Main_Data/Mono/libpxcface_c

DllNotFoundException: libpxcface_c

at (wrapper managed-to-native) Intel.RealSense.Face.FaceConfiguration:FaceConfiguration_SetTrackingMode (intptr,Intel.RealSense.Face.TrackingModeType)

at Intel.RealSense.Face.FaceConfiguration.set_TrackingMode (TrackingModeType value) [0x00000] in :0

at RawStreamsController.Start () [0x00000] in :0

-------------------------------------------------------------------------------------

In addition, I found no copy Face related DLL in runtime. (D:\Unity3d Project\RealSense Unity Project\Main_Data\Plugins\runtime)

I manually copy the DLL here also can not work. (C:\Program Files (x86)\Intel\RSSDK\runtime\face)

0 Kudos
17 Replies
MartyG
Honored Contributor III
685 Views

As a first step, please check through the steps in my guide to installing R3 in a Unity project to ensure that you have done each of the steps. Thanks!

0 Kudos
天虞
Beginner
685 Views

MartyG

C:\Program Files (x86)\Intel\RSSDK\doc\PDF\sdkdevguide.pdf

yes.I checked it, and I didn't use R2 sdk.

I submitted the bugDemo project on github. Please view. Thanks..

https://github.com/yty/RawStreams.unityBug GitHub - yty/RawStreams.unityBug: RawStreams.unityBug

Run exe. can't work.

Editor Run Work.

0 Kudos
天虞
Beginner
685 Views

1. editor play

only realsense.core = work ok...

realsense.core + realsense.face = work ok ....

2. exe play

only realsense.core = work ok...

realsense.core + realsense.face = work error ....

https://github.com/yty/RawStreams.unityBug/blob/master/Runtime/Main_Data/output_log.txt RawStreams.unityBug/output_log.txt at master · yty/RawStreams.unityBug · GitHub

0 Kudos
MartyG
Honored Contributor III
685 Views

It is night time in my time zone sadly, so I will try to look at your github on Monday morning. Good luck!

0 Kudos
天虞
Beginner
685 Views

The problem is more specific.

 

Use the following functions will have a problem, there may be more..

//moduleConfiguration.TrackingMode = TrackingModeType.FACE_MODE_COLOR; //add => have bug

//Status applyChangesStatus = moduleConfiguration.ApplyChanges(); //add => have bug

//moduleConfiguration.EnableAllAlerts(); //add => have bug

0 Kudos
MartyG
Honored Contributor III
685 Views

It is a difficult issue to diagnose. When a full Build and Run test is done in Unity (what you call an exe run), Unity accesses some functions that it does not use in the editor run. So a program may run fine in the editor but have breakages in the full Build and Run mode.

Logic would suggest that if your project works in Build and Run with the Core (Essentials) module only but fails when the optional Face module is also installed, then there is something in the Face module that Unity does not like when in Build and Run mode.

In my own R3 Unity installation, I never encountered a file called Intel.RealSense.Face.unity.dll. When I imported the Face module, the files in the project were:

Intel.RealSense.unity.dll

libpxcccore.dll

libpxcface.dll

As far as I know, these three DLL files should be all that your project needs for it to use the Face features, if you installed R3 using the steps in my guide (import the Core module into your project first and then import the Face module). .

But as I said earlier, if your Core + Face program runs fine in the editor but not in Build and Run, then that suggests that Unity is finding a conflict in the Face module when it accesses whatever extra features it uses to generate the Build and Run mode.

0 Kudos
lgre
Beginner
685 Views

Hi again,

Still get the same error with a call to ApplyChanges().

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleositeUnity5.6.0/un_air_de_famille_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleositeUnity5.6.0/un_air_de_famille_Data/Mono/.\libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleositeUnity5.6.0/un_air_de_famille_Data/Mono/libpxcface_c

DllNotFoundException: libpxcface_c

at (wrapper managed-to-native) Intel.RealSense.Face.FaceConfiguration:FaceConfiguration_ApplyChanges (intptr,Intel.RealSense.Face.AllFaceConfigurations)

at Intel.RealSense.Face.FaceConfiguration.ApplyChanges () [0x00000] in :0

at RealsenseController.Start () [0x00000] in :0

What is strange is that if I comment calls to EnableAllAlerts(), ApplyChanges() the built executable work well and I can use FaceModule data functions like faceDatas.QueryFaceByIndex(0)

but I don't think face config is applied.

Does the unity wrapper implement all SDK features ?

Any idea ?

Here is my code :

// Use this for initialization

void Start()

{

/* Create SenseManager Instance */

sm = SenseManager.CreateInstance();

/* Create a SampleReader Instance */

sampleReader = SampleReader.Activate(sm);

/* Enable Color & Depth Stream */

sampleReader.EnableStream(StreamType.STREAM_TYPE_COLOR, colorWidth, colorHeight, colorFPS);

/* Subscribe to sample arrived event */

sampleReader.SampleArrived += SampleArrived;

faceModule = FaceModule.Activate(sm);

faceDatas = faceModule.CreateOutput();

faceConfig = faceModule.CreateActiveConfiguration();

//faceConfig.EnableAllAlerts();

faceConfig.Detection.isEnabled = true;

faceConfig.Landmarks.isEnabled = true;

faceConfig.Pose.isEnabled = true;

//faceConfig.TrackingMode = TrackingModeType.FACE_MODE_COLOR;

faceConfig.Strategy = Intel.RealSense.Face.TrackingStrategyType.STRATEGY_CLOSEST_TO_FARTHEST;

faceConfig.Detection.maxTrackedFaces = 1;

faceConfig.Landmarks.maxTrackedFaces = 1;

faceConfig.Pose.maxTrackedFaces = 1;

//faceConfig.ApplyChanges();

faceModule.FrameProcessed += FaceModule_FrameProcessed;

// Initialize pipeline

sm.Init();

// Create NativeTexturePlugin to render Texture2D natively

texPlugin = NativeTexturePlugin.Activate();

// Update material's Texture2D with enabled image size.

RGBMaterial.mainTexture = new Texture2D(colorWidth, colorHeight, TextureFormat.BGRA32, false);

// Flip the image

//RGBMaterial.mainTextureScale = new Vector2(-1, -1);

// Retrieve native Texture2D Pointer

colorTex2DPtr = RGBMaterial.mainTexture.GetNativeTexturePtr();

// Start Streaming

sm.StreamFrames(false);

}

void SampleArrived(object sender, SampleArrivedEventArgs args)

{

if (cameraPaused) return;

if (args.sample.Color != null) texPlugin.UpdateTextureNative(args.sample.Color, colorTex2DPtr);

}

private void FaceModule_FrameProcessed(object sender, FrameProcessedEventArgs args)

{

if (cameraPaused) return;

faceDatas.Update();

Face face = faceDatas.QueryFaceByIndex(0);

if (face != null)

{

ldata = face.Landmarks;

if (ldata != null)

{

points = ldata.Points;

}

PoseData posDatas = face.Pose;

if (posDatas != null)

{

headPos.Set(-posDatas.HeadPosition.headCenter.x / 1000, posDatas.HeadPosition.headCenter.y / 1000, posDatas.HeadPosition.headCenter.z / 1000);

headRot = Quaternion.Euler(-posDatas.Angles.pitch, -posDatas.Angles.yaw, posDatas.Angles.roll);

}

}

}

0 Kudos
MartyG
Honored Contributor III
685 Views

I'm afraid I do not yet have enough knowledge about using the R3 SDK with Unity to be able to identify your project's error, as I have only had time to spend a couple of days with R3 before I had to return to the R2 SDK, which my own full-time project is based in.

I did some research in general on Unity's "Fallback handler could not load library" error though. A lot of people seemed to have experienced it in non-RealSense projects. People reported that their project still ran despite the errors - it was just annoying to have the debug console fill up with the messages.

A frequent cause of the error seemed to be if a Unity project was started on one PC and then transferred to another PC and the user tried to continue developing the project on that machine. Have you moved your own project between machines?

For more information on this subject, you can google for 'fallback handler could not load library unity'

A safe way to transfer Unity content (objects, scripts, etc) from an existing project into a fresh project file on a new machine, so you do not have to start from the beginning, is to package your existing project's content into an 'Asset Package'. You can then import the created package into a new project file on another PC.

https://docs.unity3d.com/Manual/HOWTO-exportpackage.html Unity - Manual: Exporting Packages

0 Kudos
lgre
Beginner
685 Views

Hi MartyG

Thx for the search and links.

I reproduced the problem with the R3 SDK unity sample Rawstream.unity project by adding these lines of code

faceModule = FaceModule.Activate(sm);

faceDatas = faceModule.CreateOutput();

faceConfig = faceModule.CreateActiveConfiguration();

faceConfig.EnableAllAlerts();

faceConfig.Detection.isEnabled = true;

faceConfig.Landmarks.isEnabled = true;

faceConfig.Pose.isEnabled = true;

faceConfig.TrackingMode = TrackingModeType.FACE_MODE_COLOR;

faceConfig.Strategy = Intel.RealSense.Face.TrackingStrategyType.STRATEGY_CLOSEST_TO_FARTHEST;

faceConfig.Detection.maxTrackedFaces = 1;

faceConfig.Landmarks.maxTrackedFaces = 1;

faceConfig.Pose.maxTrackedFaces = 1;

faceConfig.ApplyChanges();

faceModule.FrameProcessed += FaceModule_FrameProcessed;

error was :

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/rawstream/test_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/rawstream/test_Data/Mono/.\libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/rawstream/test_Data/Mono/libpxcface_c

DllNotFoundException: libpxcface_c

at (wrapper managed-to-native) Intel.RealSense.Face.FaceConfiguration:FaceConfiguration_ApplyChanges (intptr,Intel.RealSense.Face.AllFaceConfigurations)

at Intel.RealSense.Face.FaceConfiguration.ApplyChanges () [0x00000] in :0

at RawStreamsController.Start () [0x00000] in :0

Some dll seemed to be not copied in binary directory.

But I finaly got the built executable working by adding face module dlls in binary Plugin directory manually like this :

0 Kudos
MartyG
Honored Contributor III
685 Views

Thanks so much for the information on how you resolved your problem. It is sure to be useful to others who encounter this error. Best of luck with the rest of your project, and please come back to the forum any time if you need further assistance.

0 Kudos
lgre
Beginner
685 Views

I'm facing with a crash I related to in this thread MartyG.

Do you know if with R2SDK I can :

- get a 1920x1080 color image

- get head position and head rotation

- get face landmarks

0 Kudos
MartyG
Honored Contributor III
685 Views

I'm not sure I can say whether R2 is more stable than R3 with Unity, as I have only used R3 with Unity for a couple of days. I think the best way to put it would be that the RealSense tools are far more proven with R2, whereas with R3 you do not have a Unity Toolkit and so have to write your own RealSense scripts.

I recently used Unity with 1920x1080 in the 'Intel Level Up' game development contest and my RealSense F200 camera worked fine with hand and face tracking at that resolution. I have not used it for color streaming at that resolution though, so can't offer insight on how a color stream performs under that resolution setting.

0 Kudos
lgre
Beginner
685 Views

ok thx for informations

...and would you have a sample on how to use face tracking landmarks with R2 plz ?

0 Kudos
MartyG
Honored Contributor III
685 Views

The R2 SDK comes with a Unity sample called the AR Mirror that tracks the landmark points on the face and overlays virtual objects on the real-life face. The samples can be found in an application called the Sample Browser. This is in a folder called 'Intel RealSense SDK Gold'.that will be placed on your desktop when the R2 SDK has been installed.

The R2 SDK also has a script called 'TrackingAction' in its Unity Toolkit that you can drag and drop into a Unity object and configure its settings in the Inspector panel without having to do any programming. With TrackingAction, you can make objects respond to hand joint and face landmark movements.

0 Kudos
lgre
Beginner
685 Views

Ok thx for your answer

 

I'm going to adapt my project with R2.

 

I don't need objects to follow face points, I have to get some face landmarks to perform some morphing shaders.

Hope that will be more steady...

Thx

0 Kudos
lgre
Beginner
685 Views

I adapted my unity project with R2 SDK and I use face tracking landmarks and the built app doesn't crash anymore.

0 Kudos
lgre
Beginner
685 Views

Hi

I get the same problem

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/.\libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/.\libpxcface_c

Fallback handler could not load library D:/Blue Yeti/laugre/new_unity_projects/bin/paleosite/un_air_de_famille_Data/Mono/libpxcface_c

DllNotFoundException: libpxcface_c

at (wrapper managed-to-native) Intel.RealSense.Face.FaceConfiguration:FaceConfiguration_EnableAllAlerts (intptr)

at Intel.RealSense.Face.FaceConfiguration.EnableAllAlerts () [0x00000] in :0

at RealsenseController.Start () [0x00000] in :0

I tried in both 32 and 64 bits build but that's the same.

I checked in my unity binary Plugin folder and I have libpxccore_c.dll and libpxcface_idc.dll but that doesn't work.

Did you find a solution ?

Thank's

0 Kudos
Reply