Realsense Augmented Reality Sample

[GAgui]

Hi everyone I would like to know If is there some unity project sample of realsense using augmented reality?, like custom 3d model in the mesh created by the realsense r200.

I would like to have the toyz sample or the cat sample.

[MartyG]

Version 2.14.0 and newer of RealSense SDK 2.0's Unity wrapper has a 2D AR sample scene called AR Background.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/unity/Assets/RealSenseSDK2.0/Scenes/Samples/ARBackground.unity librealsense/ARBackground.unity at master · IntelRealSense/librealsense · GitHub

If you require support for older camera models, the 2016 R2 SDK for Windows has a Unity sample called AR Mirror.

[GAgui]

Thanks for the reply, I want to know how I can put 3D element in Unity in the mesh created by camera

like this https://www.youtube.com/watch?v=btwU_pT-tao&t=22s Intel ToyZ: Gameplay - YouTube

[MartyG]

Toyz is based on the old RealSense R200 technology from 2015, and a software technique called Scene Perception that can translate real-world objects into virtual ones. It was likely produced in an early RealSense SDK. The only version still available that could replicate that with an R200 is the '2016 R2' version.

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?sample_scene_perception_unity.html Intel® RealSense™ SDK 2016 R2 Documentation

https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_sp_scene_perception.html Intel® RealSense™ SDK 2016 R2 Documentation

I do not know how you could do scene perception with the current RealSense 400 Series cameras and SDK 2.0 though. Although the 400 Series uses stereo imaging like the R200 does, the software support for converting real objects to fictional virtual ones in realtime is not yet available as far as I know.

Intel's most recent use of RealSense to mix real and virtual environments was in the 2017 Project Alloy 'merged reality' headset that contained a ZR300 camera. The Alloy project was ended before the product was released though.

https://m.youtube.com/watch?v=Be-MJxeCpvU&t=206s https://m.youtube.com/watch?v=Be-MJxeCpvU&t=206s

[GAgui]

Hi MartyG, I am using a realsense R200, for that reason I use the reference toyz, I would like to use de mesh created to put my 3D element liek does vuforia, or ARKit, etc.

Thanks.

[MartyG]

I never used Scene Perception in the old SDK, so the best advice I can give is to load 2016 R2's Unity scene perception example RF_ScenePerception into Unity and pick it apart to see how it works with the help of the documentation for the sample.

Another good reference is the R200 game Tanked, which was made in Unity.

https://software.intel.com/en-us/videos/tanked-augmented-sand-game Tanked Augmented Sand Game | Intel® Software

Edit: there was also a making-of article for Tanked that included some sample scripting but it is now a broken link. I found a working version of it in Chinese and converted the text to English for you, attached as a PDF to this message.