- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm using SR300 and trying to get the correct transformation between the color camera and the depth(=IR) camera. I expected that the depth, IR and point clouds streams are all based in the same coordinate system or in the identical transformation, but I noticed that they are slightly different. Please see the following output which I got from `rs_extrinsics`.
Please see the following output which I got from `rs_extrinsics`. The transformations from the color stream to the depth/IR/point clouds are all different. Shouldn't they be the same? Which transformation should I use to do the extrinsic calibration? If anyone saw this issue before, please leave comments too.
- From RS_STREAM_COLOR to RS_STREAM_DEPTH
rotation [[ 0.99999791 0.00196758 -0.0005486 ]
[-0.001966 0.99999398 0.00285708]
[ 0.00055421 -0.00285599 0.99999577]]
translation [[-0.02549983]
[-0.00113199]
[-0.00392998]]
- From RS_STREAM_COLOR to RS_STREAM_INFRARED
rotation [[ 0.99999791 0.00196758 -0.0005486 ]
[-0.001966 0.99999398 0.00285708]
[ 0.00055421 -0.00285599 0.99999577]]
translation [[-0.02549983]
[-0.00113199]
[-0.00392998]]
- From RS_STREAM_COLOR to RS_STREAM_POINTS
rotation [[ 1. 0. 0.]
[ 0. 1. 0.]
[ 0. 0. 1.]]
translation [[-0.02549983]
[-0.00113199]
[-0.00392998]]
- From RS_STREAM_DEPTH to RS_STREAM_POINTS
rotation [[ 0.99999791 -0.001966 0.00055421]
[ 0.00196758 0.99999398 -0.00285599]
[-0.0005486 0.00285708 0.99999577]]
translation [[ 0.]
[ 0.]
[ 0.]]
- From RS_STREAM_COLOR to RS_STREAM_COLOR_ALIGNED_TO_DEPTH
rotation [[ 0.99999791 0.00196758 -0.0005486 ]
[-0.001966 0.99999398 0.00285708]
[ 0.00055421 -0.00285599 0.99999577]]
translation [[-0.02549983]
[-0.00113199]
[-0.00392998]]
- From RS_STREAM_COLOR to RS_STREAM_DEPTH_ALIGNED_TO_COLOR
rotation [[ 1. 0. 0.]
[ 0. 1. 0.]
[ 0. 0. 1.]]
translation [[ 0.]
[ 0.]
[ 0.]]
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It sounds to me like a camera calibration issue. The discussion linked to below may be of use to you.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for your comment. I suspect this is not the calibration issue because I suppose the output of rs_extrinsics is predefined in the factory based on each sensor's position and rotation (This is not the issue about the mismatch of the actual stream image, but about the mismatch of the predefined transformation matrix). What I don't get is that the depth/IR/points streams are all from the same sensor, but it shows the different extrinsics.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello kazoo_kmt,
Thanks for reaching out!
Perhaps the information in this document will be of help, it explains how the extrinsic camera parameters work. It might help you understand how the coordinate system work on librealsense:
https://github.com/IntelRealSense/librealsense/blob/master/doc/projection.md# extrinsic-camera-parameters
Pedro M.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for replying. I've already checked the document, but even it says "The extrinsic transformation between depth and infrared is the identity transform", the returned transformation between the IR and the Depth streams are slightly different.
https://github.com/IntelRealSense/librealsense/blob/master/doc/projection.md# appendix-model-specific-details librealsense/projection.md at master · IntelRealSense/librealsense · GitHub
Am I missing something?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Perhaps the difference is due to the physical distance between the IR and the depth sensors. As you may know they are a little bit separate one from the other which may translate into slight differences in their readings.
Pedro M.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Both depth and IR streams are from the same IR sensor, shouldn't have any physical distance between them.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There are two IR sensors on the SR300 as you can see in Figure 2-1 of the SR300's datasheet ( http://www.mouser.com/pdfdocs/intel_realsense_camera_sr300.pdf), the IR camera and the IR laser projector.
If you check section 4.1 of the same document you can read that "...the IR projector and IR camera operate in tandem using coded light patterns to produce a 2D array of monochromatic pixel values. These values are processed by the imaging ASIC to generate depth and/or infrared video frames ..."
So, the depth and IR frames are indeed captured by two different sensors and I believe this is the reason why of the slight offset.
Pedro M.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As Figure 4-2 and 4-3 show, both use same IR camera. The depth uses the projected pattern from IR laser projector, but when it captures the pattern, it uses the IR camera and doesn't look there are two separate sensors. I still don't understand why the difference of the image processing creates the difference in the extrinsics...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Please let me check what might be happening, I'll get back to you as soon as possible.
Pedro M.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi kazoo_kmt,
We have an update for this case.
The matrices for RS_STREAM_INFRARED and RS_STREAM_DEPTH are equal to each other because they come from the same IR sensor. They are very close to, but not exactly the same as, the matrix for RS_STREAM_POINTS because RS_STREAM_POINTS is not actually read from the sensor. It is a synthetic stream, as shown in this doc: https://software.intel.com/sites/products/realsense/camera/rs_8h.html, It can be said that RS_STREAM_POINTS is the ideal, therefore it has a perfect identity matrix. The matrices for RS_STREAM_DEPTH and RS_STREAM_INFRARED are based on real world data, therefore not perfect, but very close to ideal.
Does that help?
Pedro M.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I can't find pxcfaceconfiguration.h file. Where can I download this file?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page