R200 and UE4 plugin error -3

[HCent1]

Hello,

I'm trying to get the official Intel UE4 plugin to work with my R200 RealSense camera. So far the camera is working properly with the example applications and the plugin builds properly using the 2016 r2 SDK. I'm using a simple BluePrint for testing that only displays the color stream (taken from the plugin page at https://software.intel.com/en-us/articles/intel-realsense-sdk-plug-in-for-unreal-engine-4 Intel® RealSense™ SDK Plug-In for Unreal Engine* 4 | Intel® Software ):

The problem I have is that when I play it, I get this error in the console: RealSensePlugin:Error: RealSenseImpl::CameraThread() : SenseManager Initialized : -3

I've checked the source code of the plugin and I'm not able to identify why this is happening. I would appreciate any clues on how to go about fixing this.

Thanks!

Hector

[MartyG]

I'm not a UE4 expert, though I can read C# script language, so that's something. Looking through the code of the RealSenselmpl sample on Github, it looks like the error message relates to lines 111-120, the CameraThread() function, which initializes the SenseManager.

*********************

void RealSenseImpl::CameraThread()

{

uint64 currentFrame = 0;

fgFrame->number = 0;

midFrame->number = 0;

bgFrame->number = 0;

pxcStatus status = senseManager->Init();

RS_LOG_STATUS(status, "SenseManager Initialized")

assert(status == PXC_STATUS_NO_ERROR);

*********************

The bottom line - assert(status == PXC_STATUS_NO_ERROR); - seems to be the most significant. Your error message returns a value of -3 for SenseManager.

In 2015, Intel staffer Jonathan M had the following advice about PXC_STATUS_NO_ERROR:

"The pxcStatus is used to return some warning information along with failure information. In some cases, the function will return appropriately, but the SDK wants to warn the user/developer that something inappropriate, but not code-breaking has happened. Check the value definitions in pxcstatus.h for a listing of all the codes".

"Basically, if your function returns a negative value (pxcStatus < PXC_STATUS_NO_ERROR) means that the code should not continue, and you as a developer should include some error handling that defines how you want the app to degrade, or restart, or close. There shouldn't be any cases where a negative status value means the app is behaving correctly".

So we can interpret a couple of things from this:

* If your sample program is running then it may be a warning error rather than a fatal error.

* You should be able to look up what an error value of -3 for pxcstatus.h. Using Jonathan's advice, I managed to track down a script that lists the error code meanings.

https://github.com/getnamo/realsense-ue4/blob/master/Plugins/RealSensePlugin/ThirdParty/RSSDK/Include/pxcstatus.h realsense-ue4/pxcstatus.h at master · getnamo/realsense-ue4 · GitHub

-3 means PXC_STATUS_ITEM_UNAVAILABLE= ... basically, an item is not found or is unavailable.

Edit: it is logical to speculate that the item that cannot be found is the SenseManager component.

Hope other developers can help you further with your UE4 problems!

[HCent1]

Thank you MartyG for looking into this. I did find as much as you did and it is indeed that Init() call where the PXC_STATUS_ITEM_UNAVAILABLE error is happening, but I have no idea why. I followed everything leading to it and also tested for the camera manager (senseManager pointer) right before that call and it seems to be fine (at least not NULL). The only thing I can think of is that it seems UE4 is instantiating senseManager more than once and I think there can only be one instance of it (if I follow correctly the UE4 Intel article I posted). I get these log messages printed twice (I added them for debugging) each time I open the project so it seems it calls the RealSenseImpl constructor twice:

RealSenseImpl::RealSenseImpl()

{

session = std::unique_ptr(PXCSession::CreateInstance());

RS_LOG_STATUS(session != nullptr, "CreateInstance");

assert(session != nullptr);

senseManager = std::unique_ptr(session->CreateSenseManager());

RS_LOG_STATUS(senseManager != nullptr, "CreateSenseManager");

assert(senseManager != nullptr);

.....

Otherwise, the code really is very simple and all the example applications run fine so I'm not sure what the problem is.

Thanks!

Hector

[MartyG]

Typically, at least in my experience with the Unity engine, these checks for components create a duplicate and then destroy it without causing any problems if they find that the component already exists.

This info I found seems relevant:

"In UE4, when an object attaches a component, it receives a new instance of that component class. The same component class can be instanced many times and attached to many objects in the same project. This presents a problem for the RealSense components, however. Logically, there is only one Intel® RealSense™ camera that a developer needs to control. But because it's possible to create multiple instances of the same component class, every instance of a RealSense component will get its own copy of camera data."

"To address this issue, the RealSense Plugin uses an object called the RealSense Session Manager, a singleton that is automatically instantiated when the first RealSense component is added to an object in the game. All subsequent RealSense component instances receive a reference to the Session Manager when they are initialized. The RealSense Session Manager owns the master copy of all Intel RealSense SDK data and each RealSense Component reads from and writes to it."

[HCent1]

Yes, that's the paragraph I was referring to. Yes, I wonder if UE4 is destroying the duplicates, but still I also wonder why the "impl" class is it being instantiated if the manager is not doing it. Maybe UE4 instantiates all the components at loading time? On the other hand, if this is not the issue then I'm out of clues.

[MartyG]

Maybe under the assert(senseManager != nullptr); statement, you could insert an If instruction that says:

If (assert(senseManager != nullptr) {

return;

}

In theory, if the IF statement finds that SenseManager is not null (because an instance of it has already been instantiated) then it is told by the return statement to do nothing. Maybe that would stop if from being able to get to the point where it tries to instantiate another copy of it.

Edit: I remembered that I had some experience of editing the Unity engine's SenseManager script. So I'll post some snippets here. All of this code is Intel's, not mine.

First, in the header of the script it says:

************

public PXCMSenseManager SenseManager = null;

***********

OnEnable of the script (when the script becomes active), it creates a SenseManager instance, and if SenseManager is null then it returns.

***********

void OnEnable()

{

Initialized = false;

/* Create a SenseManager instance */

SenseManager = PXCMSenseManager.CreateInstance();

if (SenseManager == null)

{

print("Unable to create the pipeline instance");

return;

}

***********

Then later on in the script, the SenseManager is initialized.

***********

/* Initialize the execution */

_sts = SenseManager.Init();

if (_sts < pxcmStatus.PXCM_STATUS_NO_ERROR)

{

return;

}

[HCent1]

OK, so it seems that indeed those senseManager instantiations get deleted so it shouldn't be the reason why Init() is failing. I'm out of ideas now. I'm not very familiar how the RealSense SDK works yet, but I followed the code and did some more debugging by logging at different points and it seems that the manager gets correctly setup (a stream is enabled) and the device is detected with the correct model number, so not sure what else to look for.

[MartyG]

I should have asked earlier, does the error message actually stop your program from running? I'm as obsessive as the next developer about dealing with errors in the console as I hate to have them come up each time the program runs. Sometimes engines throw out messages that you just can't do anything about though and you go through the grief stages of anger, denial and finally acceptance after having tried every other thing you can think of.

For example, in Unity there is a common one that says "device ID failed" because Unity couldn't detect the plugged-in joypad. Or there's the one that talks about "repainting the GUI" when the RealSense tracking script does something that ticks Unity off. Since nothing you do to your project can prevent them, you just end up wearily accepting them if they appear and don't effect your project, and clear the console. If it's a non-fatal error and doesn't appear to impair the project, the end-user won't see if anyway when it's running outside of the UE4 editor.

[HCent1]

It doesn't crash UE4 but the RealSense camera doesn't do anything. No image is acquired and it doesn't power on.

[MartyG]

Ok, so it is more serious than just a harmless warning message then!

I went through the YouTube tutorial video for the same sample project that you created. Their blueprint is virtually identical to yours and works when they activate it. The only difference in your blueprints was that they set 'Set Color Camera Resolution' to 1920 x 1080 x 30, whereas you used the much humbler 640 x 480 x 30.

There was one more step in the video after completing the blueprint. They went to the main project editing window and added RealSense mode to the project with GameMode > Select GameMode Class > RealSenseCameraMode. Then when they hit play, the color stream appeared in their project. Does your project allow you to add the color mode, or does the SenseManager problem prevent it?

[HCent1]

Yes, I switched the GameMode too (otherwise there would be no RealSense logs at all in the console since the component wouldn't be loaded). I also tried different resolutions.

I wish there was a more clear statement from Intel on their online store. After I bought the R200 ($90 seemed like a good deal) it wasn't until I tried to use it that I found that the device is actually not supported anymore. I was directed to the driver install page, where you can find the camera manager for the R200 but the SDK offered below has no support for it (and this is not stated). I had to search online for a while to find out that SDK 2016 R2 was the last one to support it, and to download older versions of the SDK is not straight forward just by browsing the Intel site (Google helped me to find it). Even the UE4 plugin has lost support from the developer stating Intel has changed focus. I might have to switch to a Kinect for the VR work I wanted to make, but I was excited about the lightweight design and outdoor capabilities of the RealSense R200.

[MartyG]

There are new advanced RealSense cameras coming this year, such as the ZR-300 and R400. It is understandable if your purchasing confidence has been dented though.

https://newsroom.intel.com/chip-shots/intel-announces-tools-realsense-technology-development/ Intel Announces Tools for RealSense Technology Development | Intel Newsroom

[idata]

Hey VitualHC,

I have the same problem that you have! Did you solve your problem with UE and realsense pluigin?

Regards,

[HCent1]

Hi saharaseeri,

I just saw your reply after some time of not working on this anymore. Now I'm getting back to trying to achieve something with the R200 and UE4. Did you find anything since then? I'll let you know if I get it to work.

Regards

[MartyG]

Yes, please do share the news if you get something working with UE4, virtualHC. Good luck!