Items with no label
3335 Discussions

How to give the illusion of a real window?

ZSpar
Beginner
716 Views

Please check the Youtube video link.

My question maybe stupid but I wan't to replicate this solution for my basement but I'm stuck with the amount of informations, technology and language coding combination to start the project. Can you help me the understand where to start. BIG thanks for the community :)

 

https://youtu.be/InSRK0ZyLYU

 

 

0 Kudos
1 Solution
MartyG
Honored Contributor III
403 Views

Videogame development companies tend to prefer hiring graduates with math and computer science degrees rather than those who have been on videogame design degree courses, as the pure math and computer science skills make them far more adaptable employees. The specifics of applying those skills to game development can be trained quite quickly in-house.

 

Likewise, if you have employees with non-specific skills like C++ and 3D graphics then it will make it easier for them to adapt to whatever technology comes along than seeking people who specialize narrowly in one or two particular disciplines. Being a generalist will allow them to develop new tools to achieve the parts of your project that may not be possible with existing solutions.

 

Because modern RealSense cameras are extremely flexible with the hardware and software configurations that they work with (for example, the 400 Series can work with any Intel or ARM processor), they should be able to rise to most challenges that you put in front of them.

 

When thinking about future camera tech, I like to imagine three-dimensional projection in terms of something like Power Rangers SPD.

 

https://www.youtube.com/watch?v=GBW5jw93Yps

 

View solution in original post

0 Kudos
9 Replies
MartyG
Honored Contributor III
403 Views

This is similar to the 'virtual balcony' video screens that some cruise ships now have for cabins without a real balcony.

 

https://www.youtube.com/watch?v=4__AIB8Bl-4

 

I also did something like this for a PC computer game I worked on, where a virtual window in the game showed a live feed from a camera view elsewhere in the game world.

 

Unless you want the virtual window to be interactive and respond to people, it isn't so much a RealSense project as a webcam project, displaying a video feed from another location. You could conceivably use live webcam feeds from the internet, such as African plains filled with animals or an inner city, as the live image on your window. EarthCam is a good source for such feeds.

 

https://www.earthcam.com/

 

The live feeds could be handled by a 'media server' PC, which is typically a small and affordable PC that is dedicated to the sole task of controlling media such as videos, music and the web and has a design that blends in well with the room. An example of a small PC that could be used as a media server is Intel NUC.

https://www.smarthomebeginner.com/intel-nuc-htpc-media-center/

 

Conceivably, if you wanted the display to show just the area outside of your house then you could hook up a camera somewhere on the outside of your house and link it to the media server. If it were an IP Camera then the video feed could be sent wirelessly to your basement.

 

https://en.wikipedia.org/wiki/IP_camera

0 Kudos
ZSpar
Beginner
403 Views

Hi MartyG thanks a lot for taking the time to reply.

 

I understand your point, but what I'm trying to accomplish is the video background is moving in real time as the person close to it is also moving.

 

As you can se in the same video. The TrackingSensor/Camera/DepthCamera is giving the illusion that it's a real window and not a virtual one. Please let me know if its not clear. Ill do my best explain it in a different way

 

The Virtual Balcony you mentioned is just an IP camera connected to a regular screen

 

Cheers

0 Kudos
MartyG
Honored Contributor III
403 Views

I can understand what you mean. In the PC game I worked on, the player's character had near and far distance cameras attached to them so that the view of the world moved with the character. So if they moved or turned, the view of details such as the sky and sun / moon changed with them as their point of observation changed (though the world and sun / moon themselves were not being changed by the player's movements).

 

Assuming that the background only has to change for the amount of distance that it takes for you to walk from one side of the window to the other (as it would not be realistic if it kept changing when you were not in front of it) then you could perhaps make use of some kind of 'follow' body-tracking robotics. An IP camera could be attached to some kind of track that moves back-forward and left-right in response to your position in front of the mirror and your distance from it.

 

Earlier in 2019, a member of Intel gave a seminar about using the RealSense T265 Tracking Camera and a D435 depth camera to create a robot that could track and follow a person.

 

https://www.youtube.com/watch?v=62vm0_RZ1nU

 

1.png

0 Kudos
ZSpar
Beginner
403 Views

Hi MartyG, this is a sharp video, ill take the time it need to understand how I can use it for my project.

 

On my original video a Kinect is used as the tracking device combined with OpenNI Libraries to have one screen working. From then to have both screens showing the same video. One screen is sharing the QuickTime context with the OpenGL context.

 

This smart guy create an app that allows the user to specify the physical layout of the displays in relation to the tracker origin (Kinect) in a table format. This way, any gaps between the displays are automatically calculated out of the rendered views and the tracking data is now in relation to the displays.

 

Since this has been done 10years ago, Kinect devices are discontinued and technology has evolved. And I wish with your help and the help of the community you can put me on track to work on the latest device and software available to make this magic happen.

 

I have a budget for my project but since I'm not a programmer. I need to know the right Device, Software, Language (C++. C...) and the skill set needed to accomplish the job. Those informations are crucial for me to hire the right people.

 

I hope you understand my dilemma

 

Best regards,

0 Kudos
MartyG
Honored Contributor III
403 Views

A recent development in use of RealSense with adaptable display walls was a technology called Brixels.

 

https://www.intelrealsense.com/brixels-powered-by-intel-realsense-technologies/

 

Outside of RealSense, and at the higher end of the budget scale, Intel have also been using 3-dimensional pixels called voxels (Volumetric Pixels) with a multi-camera system they developed called True View. It uses multiple cameras to record a scene such as an American Football game or a movie set from all angles. The viewer can then view a scene from any angle as though they are directing the cameras.

 

https://www.intel.com/content/www/us/en/sports/technology/true-view.html

 

https://www.youtube.com/watch?v=8lbM5vsqJd8

 

If having the latest tech is important to you then it may be useful to wait and see what kind of specification emerges for Intel's next line of RealSense cameras. Although no pricing, release-date window or any other details are available for those products right now, my expectation (not certain knowledge) is that prices will not likely go much above the $200 level that models such as D435i are in.

 

In terms of software, I would expect C++ or Python skills to be most useful at the start of the new generation if its SDK is an evolution of the current one. Again, there are currently no specifics available. I would also expect anyone who can program the current RealSense SDK 2.0 well to have transferable skills when the next series' SDK comes.

 

If you need to get started with your project sooner rather than later, the 400 Series is still an excellent choice, as all models of RealSense cameras are tech-supported in these support channels indefinitely, and the open-source nature of its development environment means that the SDK software can keep getting updated by its community.

0 Kudos
ZSpar
Beginner
403 Views

Hi MartyG

 

WAAAAW is the best word to describe how this new technologies combined will change our world.

 

At a smaller scale do you think a C++ and Python programmer can use the RealSense camera to develop the app that will fit my needs in therm of realistic illusion. Do you think they need other skills like maths, 3D, depth camera knowledge...etc. I'm just saying words like that and not sure how they can be practically used by the X team I'm trying to put together.

 

Sorry If my questions my sounds dump but the 3D world is very new for me, and hope I can learn it fast.

 

Best regards,

0 Kudos
MartyG
Honored Contributor III
404 Views

Videogame development companies tend to prefer hiring graduates with math and computer science degrees rather than those who have been on videogame design degree courses, as the pure math and computer science skills make them far more adaptable employees. The specifics of applying those skills to game development can be trained quite quickly in-house.

 

Likewise, if you have employees with non-specific skills like C++ and 3D graphics then it will make it easier for them to adapt to whatever technology comes along than seeking people who specialize narrowly in one or two particular disciplines. Being a generalist will allow them to develop new tools to achieve the parts of your project that may not be possible with existing solutions.

 

Because modern RealSense cameras are extremely flexible with the hardware and software configurations that they work with (for example, the 400 Series can work with any Intel or ARM processor), they should be able to rise to most challenges that you put in front of them.

 

When thinking about future camera tech, I like to imagine three-dimensional projection in terms of something like Power Rangers SPD.

 

https://www.youtube.com/watch?v=GBW5jw93Yps

 

0 Kudos
ZSpar
Beginner
403 Views

Can't say enough thanks for your time. Ill work with that and see If I can get something good out of it. Ill text you once we start the design :)

 

Have a great day

0 Kudos
MartyG
Honored Contributor III
403 Views

​You are welcome to return to the forums any time. Have a great day too! :)

0 Kudos
Reply