What is the best application for using the r200 to scan full body objs for video game implementation?

[JDAng2]

Any suggestions? I have tried easy3Dscan and itSeez3D. easy3Dscan wont connect without failing errors and itSeez3D isnt scanning to the r200's maximum resolution. Any goto apps for this sort of thing I should be aware of?

Thanks so much!!

PS I am using the Unity Game Engine.

[MartyG]

Uraniom can scan peoples' heads into full-body videogame avatars and is supposed to be R200 compatible according to its makers.

http://www.uraniom.co/ Uraniom - Universal 3D avatar for VR/AR apps and video games

If you click the 'Developers' link at the top, it jumps to a section where it shows that Uraniom can be integrated with Unity.

Edit: I've gone "off shift" for the day now but it you leave a comment here then I can resume with you tomorrow morning. Thanks!

[JDAng2]

First let me say, I have asked now two questions on this forum and you have taken the time to answer both of them. Your time is greatly appreciated thank you so much!

Uraniom looks super powerful and I am looking forward to learning more about what it has under the hood.

As for maybe a bit more specifics of what I am looking for for my prototype, I am implementing the most photo realistic full body scans possible into real footage (FMV) then rigging the character. ReconstructMe is another app I have found however it is yet another app that crashes to errors while connecting. Troubleshooting that is certainly a different thread entirely but once I have figured out how to get these apps working I will repost my progress should someone else be interested in achieving the same content.

As of now I can say itSeez3D launches without errors but the renders are very sub par (at least for my needs.) If someone needs a free full body scanning app for the R200, itSeez3D is the only one I can recommend as of this comment.

Thanks again so much MartyG!

[MartyG]

You're very welcome. I'm glad I could help..

I believe ReconstructMe worked up until the '2016 R1' RealSense SDK and then had problems in subsequent SDKs such as R2 and R3, as did other features such as Facerig face model scanning and Unreal Engine compatibility (the UE driver was developed by an individual who no longer had time to update it for new SDKs).

Version 2016 R1 and those before it have been withdrawn from availability due to potential security vulnerabilities, so using an earlier SDK with ReconstructMe is unfortunately not an option.

itSeez3D is a program I have recommended to people myself.

I believe that the new ZR300 camera provides superior 3D scanning results to the R200, as it is newer camera technology. However, it only works with Linux at present, so you would probably have trouble finding scanning software that would work with it.

You may be interested in my own full-body avatar project in Unity, which takes a different approach. I hand-sculpt the body from individual objects and put the animation controls in the invisible bone points, which allows for enormously flexible real-time controllable animation.

https://www.youtube.com/watch?v=T_V8SNwLTrY 'My Father's Face' Tech Trailer 7 - YouTube

By placing live animation controls in bone points in the face and body, my avatar tech - which is now considerably more advanced than the one in that test video - has been able to achieve animation control of a level similar to this clip from the animated movie 'Sing'.

https://www.youtube.com/watch?v=enuprdVo7GA SING song "Don't You Worry 'Bout A Thing" by Meena - YouTube

I have also experimented in the past with RealSense controls in the bones of a rigged avatar like the one that you wish to create. This was a viable option too, as moving the bones with the camera twisted and stretched the model at that bone point and made it appear as though the limb was moving. The results are more unpredictable than animating the body with individual objects though.

[APetr19]

Hi reddjinnpro!

I saw you questions about human scanning with RealSense R200, and decided that maybe I can add a few comments here. I was one of the original developers of the itSeez3D scanning app, so let me try to clarify some details.

First of all, would be very nice to know what exactly you mean by "itSeez3D isnt scanning to the r200's maximum resolution". itSeez3D app was designed to work at maximum available resolution of the sensor, for both depth and RGB cameras. On top of that, the app gathers a lot of keyframes and online poses, and then stitches them with offline optimization to provide more accurate models.

Here are examples of the models created with R200 & itSeez3D: https://sketchfab.com/models/49fb5375219545e39e3586bfe6d15a7e Dmitry by itSeez3D - 3D model - Sketchfab https://sketchfab.com/models/ee59fff223ee4a6185f11a8f94746ea8 Ilya by itSeez3D - 3D model - Sketchfab

If you're getting similar quality results and this is not sufficient for you, it's probably fair to say that you're limited more by R200 hardware rather than scanning software. The depth resolution that is available won't give you much more details in 3D, and the textures are also maximum resolution R200 allows to capture.

If your results are worse than this sample model I highly recommend you reach our support. You may have issues with the scanning setup, or possibly with some of the SW settings, I hope we can diagnose this.

Actually, in both cases feel free to write to mailto:support@itseez3d.com support@itseez3d.com, or me directly alexey.petrenko -at- itseez3d.com. Or we can just continue the discussion here

One important remark to add: Uraniom made a great SW when it comes to integration with games, but as far as I know they didn't have their own scanning solution. On most demonstation with Intel they used itSeez3D app to scan people, and then their software attached head to the body and inserted this avatar into a game. Although I am not sure what they're doing now, it is possible that they added something on top of that. Excuse me here if I am mistaken.

On top of what I already said, I encourage you to try our new technology for generation of game avatars. It is called Avatar SDK and allows you to generate 3D human head with texture from a single selfie, without any 3D sensors: https://itseez3d.com/avatar-sdk.html 3D Avatar from Selfie for Games, VR and AR | itSeez3D

We use trained deep learning model to infer the shape of the head from the photo. There's Web API and Unity Plugin too. Please register here to get a dev account: https://accounts.itseez3d.com/developer/signup/ itSeez3D The API is free for demo and trial use.

Here is the kind of result you can expect with an iPhone selfie: https://sketchfab.com/models/ee7a2733cc794d98a8e1eab647bb1d90 3D full body avatar | itSeez3D Avatar SDK by itSeez3D - 3D model - Sketchfab Ofc, head is generated and the body was added later here.

Looking forward to hear more about your usecase, and really hope we can help you!

[JDAng2]

This is hands down the best forum experience I can recall ever having. I think I'm going to really enjoy working in this community. Thank you so much apetrenko.nn for taking the time to join in the discussion. You and MartyG have helped me a ton and very quickly. Since my last post I have spent nearly 15 hours tweaking the overall scanning techniques of the r200 but more so the lighting of the space around the subject and found that itSeez3D is the best tool available. There just aren't any usecases similar enough to my project to pull from online which is a good thing for originality but makes finding the best tool sets some leg work.

I will now dive into your Avatar SDK and see where it goes and post my findings here should it help others. Yourself and your team have done a stellar job with these apps and your time is greatly appreciated!

[APetr19]

No problem, please feel free to reach to our team at any time, we will be glad to help.

Avatar creation is a new business for us too and we are eager to receive feedback from all our users, so we're looking forward to hear from you later! Good luck!