Touchless main menu buttons (SR300)

[MMoha17]

Hi,

I have an application that has a main menu and would like to control the buttons using hand or face tracking. I referred to a post ( https://software.intel.com/en-us/forums/realsense/topic/587339 Unity TIp: Creating a RealSense-Powered Menu In Unity ) and was able to complete it, but I am unsure how to implement it on the buttons to go to another scene. May I know if anyone knows how I can do it?

Thank you.

[MartyG]

I'm glad you found the Unity guide useful!

Loading in a scene when a button is activated involves a simple one line instruction called Application.LoadLevel ("name of your scene file");

https://docs.unity3d.com/ScriptReference/Application.LoadLevel.html Unity - Scripting API: Application.LoadLevel

It is important to remember that you need to program Unity to wait for an input before activating the instruction, otherwise it will activate the scene load as soon as your program starts instead of waiting for an input from you.

One way to do this is to create an OnMouseDown() type function in your scripting. This tells Unity to wait until a mouse button has been clicked before activating the scene load.

public void OnMouseDown() {

Application.LoadLevel ("name of your scene file");

}

Since you are creating camera controls for your main menu though, it is preferable to use a SendMessageAction component to send the activation signal to a script containing your Application.LoadLevel instruction when a particular gesture is detected. If I remember my menu guide correctly, I believe that I had set it to activate a highlighted option when the Hand Closed gesture is made.

Please feel free to ask here if you have more questions. Good luck!

[MMoha17]

Hi MartyG,

I have managed to use hand tracking to make 1 button work but I cannot get the same Menu Selector to trigger more than 1 button. The script shown below allows only the "start game" button to work but not the "quit" button. The OnTriggerEnter() is used twice. Thank you.

public void OnTriggerEnter(Collider other)

{

Debug.Log("Hello");

SceneManager.LoadScene("game");

}

public void OnTriggerEnter(Collider other)

{

Debug.Log("Quitting...");

SceneManager.LoadScene("game1");

}

[MartyG]

I had a look through my own menu system, which uses the design in the guide, to refresh my memory about how to select multiple options with the same Hand Closed Gesture.

STEP ONE

Highlight the button that you want to add an activation mechanism to. Go to 'GameObject > Create Empty Child' to child-attach an invisible Empty GameObject to the button.

STEP TWO

Highlight the newly created Empty GameObject. Go to 'Component > Add > Physics > Box Collider' to add a Box Collider to the Empty GameObject. Then play with the collider's X, Y and Z size values until you have defined a collider that is a bit larger than your option selector object, so that the selector is enclosed totally by the collider when it enters the field.

Remember to place a tick in the collider's Is Trigger box so that Unity treats it as a trigger field.

STEP THREE

Place a SendMessageAction component in the Empty GameObject and un-tick it so that it is dormant by default. Also set it to activate when the Hand Closed gesture is detected.

STEP FOUR

Create a trigger script in the Empty GameObject. It should include the following OnTriggerEnter and OnTriggerExit instructions.

public void OnTriggerEnter(Collider other)

{

GetComponent().enabled = true;

}

public void OnTriggerExit(Collider other)

{

GetComponent().enabled = true;

}

When the selector object enters the Empty GameObject trigger field, it will automatically enable the SendMessageAction component. When the selector exits the field, the SendMessageAction will be disabled again. What this does is ensure that the action linked to this particular SendMessageAction can only be triggered when the selector is over that option. When the selector is outside of the field whilst over another option, the SendMessageAction cannot respond to the Hand Closed gesture, and so the action linked to that SendMessageAction cannot accidentally trigger.

STEP FIVE

In your trigger script, create a fire-once function with a name of your own choosing, such as TitleExit_Handconfirm. This function can only activate when an activation signal is specifically sent to it, and will be ignored otherwise. For example:

public void TitleExit_HandConfirm ()

{

TitleScreenGUI_Confirmation.GetComponent ().enabled = true;

}

What this example does is to activate a canvas GUI that has a "Are you sure you want to exit?" message with Yes and No buttons. You can put your own instructions inside the function to state what you want to happen when the button is activated.

STEP SIX

Go to the SendMessageAction and put in the 'Function Name' text box the name of the function that contains your activation instructions ( e.g TitleExit_HandConfirm ). Remember that you do NOT need to put the name of the script in this box. You are instead stating the name of a function that is inside a script that is inside the same object as the Empty GameObject. This is one of the most important things to remember - whatever function you are activating, its script must be in the same object as the SendMessageAction.

STEP SEVEN

Run your project, move your selector over the button that has your trigger, and close the hand to send an activation signal to the function that you have specified.

STEP EIGHT

Finally, repeat the above process for each of your buttons, giving each Empty GameObject its own box trigger field, SendMessageAction and a trigger script that will make something happen when the Hand Closed gesture is made whilst the selector is hovering over that button.

[MMoha17]

Hi MartyG,

Thank you for this detailed explanation.

[CCH]

I'm already follow until step 4.

If i want make my "Play" button can load my another scene, what should i write in my trigger script ?

[MartyG]

Put this in your trigger script. Put the name of the scene file in the quotation marks.

Application.LoadLevel ("name of your scene file");

[CCH]

so this will ok ?

public void PlayButton()

{

Application.LoadLevel ("game");

}

[MartyG]

Yes, if your scene file is called "game" then that will be fine.

[CCH]

i got this error when run project, suddenly my project paused after run

"Null Reference Exception : Object Reference not set to an instance of an object"

can help to fix my problem ?

[MartyG]

Can you post the complete error message please?

[MartyG]

I suspect that the problem is that when you added an instruction to load another scene, you have not added that scene to the Build window of Unity and so Unity does not know how to find it.

Open your 'game' scene file (the one that the LoadLevel instruction refers to). Then go to File > Build Settings and click the 'Add Open Scene' button. Your 'game' scene will be added to Unity's build list so it knows how to find the file when your click your scene loading button.