Items with no label
3335 Discussions

How to Track people between Multiple Cameras

NPhan1
Beginner
1,795 Views

I would like to write an app to track people using multiple RealSense cameras in a room (I have 2 R200 and 1 R435). I also want real world coordinates as part of the tracking data. Has anyone tried this before? I would really appreciate some pointers on how to achieve this.

I was thinking of using a facial recognition sample app as a starting point. But the FaceData interface does not offer any coordinate information. I also don't know how to get the app to identify the same person that walks through the field of view of multiples cameras.

Thanks in advance for your help,

NP

0 Kudos
6 Replies
MartyG
Honored Contributor III
314 Views

If it were possible to achieve, I would guess that it could be done so by overlapping the field of view of cameras to provide one large FOV, as done by a robotics expert who overlapped three R200s to create a 150 degree FOV.

Your two R200s may be most suitable for doing this. Including the D435 in the mix could be problematic in terms of programming an app, since the R200 and D435 use different SDK software that are not compatible with each other. For example, the R200 and D435's SDKs use different instructions for calculating real-world coordinates.

If a single overlapping FOV cannot prevent a person from going 'out of camera' when they pass between one camera and the next, a way around this may be to share the same database file between the cameras so that as the person leaves the view of one camera, their ID is re-acquired when they enter the view of the next camera thanks to it having access to the shared database of registered IDs.

0 Kudos
NPhan1
Beginner
314 Views

Thanks, MartyG. Is there anyway to get position data from the face tracking feature? Either camera coordinates or real-world coordinates?

0 Kudos
MartyG
Honored Contributor III
314 Views

There is a sample program for R200 that can display the 2D X and Y coordinates of the face.

https://software.intel.com/en-us/articles/intel-realsense-depth-camera-r200-code-sample-face-tracking Archived - Intel® RealSense™ Depth Camera R200 Code Sample – Face Tracking | Intel® Software

0 Kudos
NPhan1
Beginner
314 Views

Thanks, MartyG. But my ultimate goal is to get the tracked face's real-world 3D coordinates. How can I use the 2D coordinates to get there?

I found the QueryHeadPosition function in the SDK 2016 R2 Reference Manual, which looks promising because it returns the 3D coordinates. However, the documentation lists "Face Pose Data [+JS, F200, SR300]. Does this mean that this function will not work with the SR200 or D435 cameras, and that it is available in JavaScript only?

0 Kudos
MartyG
Honored Contributor III
314 Views

On a discussion by someone else who wanted world coordinates on R200, there is a sample script below for getting the world coordinates from the colour image with ProjectColorToCamera

0 Kudos
NPhan1
Beginner
314 Views

Thanks very much, MartyG. I'll give that a try.

0 Kudos
Reply