Detect body while camera is moving

[tlian5]

Hi, I just wondering if the RealSense camera can detect human boday while the camera is moving - if I put the camera on a robotic platform, can the camera detect human body while the platform is moving? Also can the camera detect while both camera and human body are moving?

Thanks.

[MartyG]

As long as the camera can see the face landmarks or body joints then yes, it can maintain tracking, although movement does increase the chances that the camera will lose sight of a tracked point and stall the tracking. In my own RealSense project in the Unity game engine with a full-body avatar, the player moves around a lot in front of the camera. As long as you do not stray too far from the center-point of the camera's view then it can maintain tracking reliably.

In that example of course, it is the body moving in front of the static camera rather than a static body being in front of a moving camera. But from the camera's perspective, it amounts to the same thing - when the body moves to the side, the camera's view of the body changes as though it is the camera position that has moved to the side.

I would think that as long as camera and body remain relatively aligned then tracking should also be able to be maintained if both are moving at the same time, though that increases the chances of a stall. Like a soccer goal-keeper diving one way to try to stop the soccer ball whilst the ball swings into the opposite side of the goal.

What you are aiming to achieve reminds me of the TV cameras in the newsroom of the United Kingdom's BBC. They stopped using human camera operators in the studio and built a track that robot TV cameras travel around the studio on, tracking the news-reader's position. They have been known to occasionally have a mind of their own though.

https://www.youtube.com/watch?v=plnGb6SlA0Q BBC News: Behind The Cameras - YouTube

[tlian5]

Thanks for your reply, I was using Kinect v2 but had an issue while the camera is miving it easily lost track of body. And Ive been told from MSDN that Kinect is not designed for moving so they dont support tracking human body while the camera is moving even I thought moving camera and moving body are the same thing.

So Im thinking convert my project to use RealSense but I would like some confirmation that the RealSense support tracking body while it is moving.

Thank you.

[MartyG]

You're very welcome. Here's a test video of my project from a few months ago, showing control of a full-body avatar with an old F200 RealSense camera from 2014. Performance would likely be even better with a modern camera model like the SR300.

https://www.youtube.com/watch?v=T_V8SNwLTrY 'My Father's Face' Tech Trailer 7 - YouTube

[tlian5]

Thanks for the video, it looks very good.

What I really want to know is : while the R200 camera is shaking or moving, the performance of person tracking feature from SDK. Does the 'Int32 npersons=ptd.QueryNumberOfPeople();' method still functional while the camera is moving, is there any delay or lost track?

Is this R200 camera is designed for person tracking while it is moving or the person tracking will only works when the camera is stationary.

Thanks.

[MartyG]

Given that the R200 was released at the start of 2015, its vision tech is aging now, The new ZR300 camera has superior motion tracking abilities, though at present you have to use it with the Linux instead of Windows. Person Tracking is supported with the ZR300 through the RealSense SDK For Linux.

From the Intel online store's feature list for the ZR300:

Having said that, the R200 has been successfully used with Person Tracking for people that are in motion (which is pretty much the same as the camera being in motion whilst the person is stationary).

https://www.youtube.com/watch?v=NyF7QaYOzA4 Intel Realsense User Initialization and Tracking - YouTube