- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Camera Model: SR300
Firmware Version: firmware': '3.21.0.0'
Operating System & Version: macOS sierra 10.12.6
librealsense version: 1.12.1
pyrealsense version: 2.1
Required info:
I am trying to get the depth map using intel realsense SR300. However i am not getting the depth map for the black pixels in an image. example: for my hair i am getting 0 value in the depth map.
I am wearing a stripes t-shirt having grey and red colour. Depth map for red stripe is generated however for the grey colour the depth map is not created. How to solve this problem? Even the depth of my hair, beard or anything black is not coming.
CODE:
import os
import cv2
import pyrealsense as pyrs
import numpy as np
def create_directory(directory):
if not os.path.exists(directory):
os.makedirs(directory)
return 1
else:
print(directory, '- already exists')
return 0
def check_directories():
try:
create_directory('./data')
create_directory('./data/depth')
create_directory('./data/color')
create_directory('./data/cad')
create_directory('./data/dac')
except:
print("Unexpected error:", sys.exc_info()[0])
return -1
return 0
def main():
file_structure = check_directories()
if file_structure == -1:
print('\nERROR: Directories can\'t be created, error thrown')
return -1
else:
print('\nDirectories created successfully...\nLaunching camera module...')
# Fire camera & launch streams
serv = pyrs.Service()
serv.start()
cam = serv.Device(device_id = 0, streams = [pyrs.stream.ColorStream(fps=10),
pyrs.stream.DepthStream(fps=10),
pyrs.stream.CADStream(fps=10),
pyrs.stream.DACStream(fps=10),
pyrs.stream.InfraredStream(fps=10),
pyrs.stream.PointStream(fps=10)])
scale = cam.depth_scale * 1000
print "scale = ", scale
# Some important variables
flag_save_frames = True
file_num = 0
# Start fetching Buffer
print('Starting Buffer...')
while(True):
cam.wait_for_frames()
current_color = cam.color[:,:,::-1]
# print current_color.shape
current_depth = cam.depth * scale
current_cad = cam.cad[:,:,::-1]
current_dac = cam.dac * scale
print "max depth", np.max(current_dac)
print "print depth val = ",np.max(cam.dac)
cv2.imshow('Color',current_color)
cv2.imshow('Depth',current_depth/1000)
cv2.imshow('CAD',current_cad)
cv2.imshow('DAC',current_dac/1000)
if flag_save_frames:
num = format(file_num, '08')
cv2.imwrite('./data/depth/' + str(num) + '.png', cam.depth)
cv2.imwrite('./data/color/' + str(num) + '.png', current_color)
cv2.imwrite('./data/dac/' + str(num) + '.png', cam.dac)
cv2.imwrite('./data/cad/' + str(num) + '.png', current_cad)
file_num += 1
k = cv2.waitKey(1)
if k == ord('q'):
print('Q Pressed...\nEnding execution')
break
if k == ord('f'):
if flag_save_frames:
print('F Pressed...\nStopped fetching frames...')
flag_save_frames = False
else:
print('F Pressed...\nStarted fetching frames...')
flag_save_frames = True
XXXWHITESPA...
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Causes of an image being black like this can include:
1. The light source in the room is too dark or too bright. The camera has a component called an IR emitter that assists the IR sensor with exposure, and if the lighting is too low or too high, the emitter can cause the IR sensor to become saturated. The IR emitter can be turned off with scripting in Librealsense.
* Enables / disables R200 emitter
*/
public void setEmitterEnabled(int value) {
setOption(RealSense.RS_OPTION_R200_EMITTER_ENABLED, value);
}
2. The 'gain' value may be too low. Increasing gain can make an image brighter.
Also, if you have a problem with depth scans that are too noisy, using a 'point cloud' type of depth scan may yield better results.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
MartyG the light source is good enough. It is always the case that the depth for the objects black in colour is undefined. Are black objects a limitation for SR300?
The depth images are very noisy and so are there any predefined methods that helps us reduce noise?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Further research revealed that it is apparently a fundamental principle of 3D scanning in general (not just RealSense) that black / dark objects do not scan well because the color absorbs the laser. A tip that was shared many times for scanning black objects was to cover them in a spray or a powder fine enough to spray on, such as foot powder or baby powder so that the laser can see the surface.
I recently found a link to a professional 3D scanning spray.
https://3dscanspray.com/ 3DScanSpray.com
In your case, I think I would think that coating your t-shirt in baby powder would be a nicer option than foot powder!
As I mentioned above, doing a 'point cloud' depth scan may provide better results if you have noise on your scan.
https://software.intel.com/en-us/articles/intel-realsense-technology-and-the-point-cloud Intel® RealSense™ Technology and the Point Cloud | Intel® Software
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi MartyG Thanks for your response.
R200 camera uses a similar technology. However, such a problem is not present there. I had my friend using it and he was able to get the depth maps for the black objects as well.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I did further research into scanning black objects, and every information source repeated the earlier suggestions of changing the lighting conditions or using a spray / powder. The only new tip I did see suggested was to turn the camera 90 degrees to change how the laser light falls onto the object being scanned.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As I noted before, shiny and/or transparent objects have a similar problem (in this case, the light is not absorbed, but reflected or refracted in a direction not visible to the camera). Hair is yet ANOTHER problem: it's just too fine to reflect the dot pattern in a coherent way, and often individual hairs are below the camera resolution. Of course if the hair is black you have more than one problem. Edges of objects are also often not picked up well as they are at an extreme angle with respect to the camera (and may be shadowed from one camera or the other), and also pick up relatively small amounts of light from the projector. As an added complication, objects that are "glossy" tend to act more and more "shiny" (more specular, or mirror-like) at glancing angles.
These are not RealSense problems per se but general problems with depth sensing.
Unfortunately using a spray or otherwise modifying the object to be scanned (eg using a hair cap) is not always feasible. Your best strategy is probably to make your algorithms robust to holes in the data, and using hole-filling algorithms were appropriate. Don't forget you also have color pixels even where there is no depth so you can peek at them to see if a hole is a "real" hole or maybe due to the pixel being dark (which you can easily determine from the color channel).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Do you know if new generation of realsense D415 & D335 deal better with this problem?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Whilst the new cameras would still be subject to the same general physics problem of dark / black colors absorbing laser light, they do have a new Advanced Mode that enables users to have greater control over depth generation and color correction.
https://github.com/IntelRealSense/librealsense/blob/development/doc/rs400/rs400_advanced_mode.md librealsense/rs400_advanced_mode.md at development · IntelRealSense/librealsense · GitHub
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page