No RGB stream available (D435)

[Ncare]

I added this comment on a previous thread ( ) but thought it might get lost, and it's an issue I'd like to solve - my apologies for the double post.

My D435 has no colour sensor available. I'm running it on a NUC 5i7RYH with Ubuntu 16.10, and have tried all four USB3.0 ports. Realsense viewer shows no colour stream; if I run the rs-sensor-control example it only shows the stereo depth sensor on the device, no RGB camera.

This same machine has run all previous iterations of the RealSense technology pretty well. Using the newest SDK with an older SR300 works fine, so I don't think it's the software or uvcvideo. It could be a power problem, but I thought USB3.1 was supposed to have the same power draw requirements as USB3.0?

Possibly related issue: when viewing sensor streams using the included /example/rs-sensor-control, choosing an IR stream causes a segmentation fault (though I can view IR output using my own apps and the realsense viewer).

CPU usage: When streaming depth only, if I use highest depth resolution the CPU (for a single core) spikes to 100%, but using a lower resolution it sits between 40-70%. I've tried compiling with OpenMP both on and off, and neither setting made a difference to RGB stream availability or IR crashes.

edit: Just noticed that when I pull up the realsense viewer it says the source is an Intel Realsense 430 (not 435) ???

[MartyG]

Although I do not have a D435 to test with, a search for Viewer pictures on Google Images shows that the camera is usually displayed as ''Intel RealSense 435'. The D430 (the circuit board camera module inside the D435 casing) does not have an integrated RGB sensor, though it is a part of the D435's casing innards. So if your camera is being detected as a D430 configuration, that might be why RGB is not made available (this is just a theory though).

[Ncare]

Yes, that seems very likely! Not sure what to do about it, unfortunately.

[MartyG]

In the Viewer's device information option just below the displayed camera name, there is an option to reset the camera hardware. You could give that a try.

[Ncare]

Resetting the hardware didn't fix the issue, but it did throw up some interesting errors about holding onto frames. So I re-patched uvcvideo (since that's usually the culprit for camera issues) and remade the librealsense build.

This did not fix the problem (still D430, still no RGB), but now if I reset the hardware through realsense-viewer I'm at least getting different, more informative errors:

16/03 17:38:41,113 WARNING [139736904591104] (sensor.cpp:313) Unregistered Media formats : [ UYVY ]; Supported: [ ]

16/03 17:38:47,457 ERROR [139736819689216] (types.h:178) get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such device

16/03 17:38:47,457 ERROR [139736819689216] (error-handling.cpp:68) Error during polling error handler: get_xu(...). xioctl(UVCIOC_CTRL_QUERY) failed Last Error: No such device

16/03 17:38:48,457 ERROR [139736819689216] (types.h:178) Cannot open '/dev/video0 Last Error: No such file or directory

16/03 17:38:48,457 ERROR [139736819689216] (error-handling.cpp:68) Error during polling error handler: Cannot open '/dev/video0 Last Error: No such file or directory

Following through on these:

- Trying to open the RGB directly with v4l2-ctl also gives "/dev/video0: No such file",

- the camera is not being registered/used by uvcvideo, according to lsmod.

- v4l2-ctl-dev is the latest version (1.10.0-1)

- lsusb gives the following:

Bus 001 Device 002: ID 8087:8001 Intel Corp.

Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Bus 003 Device 018: ID 8086:0ad4 Intel Corp.

Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

Bus 002 Device 005: ID 8087:0a2a Intel Corp.

Bus 002 Device 004: ID 04f2:0939 Chicony Electronics Co., Ltd

Bus 002 Device 003: ID 413c:2107 Dell Computer Corp.

Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

When I plug in the realsense and check dmesg I get the following:

USB Video Class driver (1.1.2.realsense-1.2.0)

[24785.814215] usb 2-1: new high-speed USB device number 12 using xhci_hcd

[24785.942911] usb 2-1: New USB device found, idVendor=8086, idProduct=0ad6

[24785.942913] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0

[24785.942914] usb 2-1: Product: Intel(R) RealSense(TM) 430

[24785.942915] usb 2-1: Manufacturer: Intel(R) RealSense(TM) 430

[24785.943511] uvcvideo: Found UVC 1.50 device Intel(R) RealSense(TM) 430 (8086:0ad6)

[24785.944351] uvcvideo: Unable to create debugfs 2-12 directory.

[24785.944410] input: Intel(R) RealSense(TM) 430 as /devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1:1.0/input/input38

So not yet solved, but getting closer to the source. Also there's a possibly related problem here: https://github.com/IntelRealSense/librealsense/issues/1086 Frames didn't arrive error - after improper shutdown · Issue # 1086 · IntelRealSense/librealsense · GitHub

I'll keep plugging at it.

edit: Other people seem to have had the same errors with various streams, there is some indication it is a kernel patch problem.

[MartyG]

Thanks for the great detailed feedback _ it is sure to be helpful for Intel support team members when looking at your problem.

When I get a wrong camera name displayed in the Viewer, I find it can help to unplug the camera from the USB port, wait a couple of seconds and then push it back in firmly all the way in.

[Ncare]

Unfortunately unplugging and replugging does not help - but it's starting to look more and more like this might be related to a known firmware bug, cf

https://github.com/IntelRealSense/librealsense/issues/1131 Frames not appearing and Camera unresponsive on realsense viewer and ros wrapper · Issue # 1131 · IntelRealSense/libreals… so hopefully it's being worked on! I will keep an eye out for any updates.

[PSnip]

Hi Carey,

I have seen stability issues with D435 on various platforms. However, nothing exactly which matches the behavior you mention. I am not sure if this will really help you, but I would have tried following

(1) Cross check all Linux Kernel patches are in place.

(2) run "rs-enumerate-devices"

- This will give you a device name. What is the name associated with your device?

- What is the firmware version?

(3) run "ls /dev/vid*", before and after you plug-in your device.

- For D435, if everything is ok, you will see 3 new device nodes being created

- 2 device nodes are for stereo IR cameras, and 3rd is for RGB device

(4) I have seen an issue with D435 where at some times it is not enumerated properly. But if you remove & replug it, it may work. For me, at times it require process be repeated 2-3 times.

(5) My D435 is more stable on PC, and less stable on embedded board I use. This could be because of either Power issue, or may be some kind of noise on USB lines. I am not fully sure.

- But connecting D435 via external USB hub seem to have helped

- Though I have not yet tried it, but I think using a shorter USB cable will help too. The cable which came with the D435 is rather too long.

Regards,

PS

[Ncare]

Hi PS,

Unfortunately none of these steps added much information. I've checked the kernel patches several times, and other RealSense devices are working ok.

Output of rs-enumerate-devices gives same info as before (notably, only the board name 430 rather than the full device name 435). Firmware version as below:

Device info:

Name : Intel RealSense 430

Serial Number : 746612070042

Firmware Version : 05.08.15.00

Physical Port : /sys/devices/pci0000:00/0000:00:14.0/usb3/3-1/3-1:1.0/video4linux/video0

Debug Op Code : 15

Advanced Mode : YES

Product Id : 0AD4

After connecting the D435 there are only ever two device nodes in use (dev/video0 and dev/video1) which are presumably the IR nodes

Obviously I've disconnected and reconnected the hardware several times now, and rebooted, nothing changes. I did try an external hub just on the off-chance it had a more stable connection, but no luck.

I will see if i can find a shorter connecting cable, and maybe try it on another CPU, to see if it is a power issue with the USB3 input ports of the NUC.

[PSnip]

Hi Carey,

I am not sure what should be the next debug step for you. Since you mention that only 2 device nodes are getting created when you plug in the device, so RGB sensor may be really missing in the signal chain. I think there are several possibilities

(1) You indeed have a device without RGB Sensor. On the D435, you can see two stereo IR sensors [these are identical and easy to spot], one IR projector [this is in middle of two IR sensors], and a RGB sensor [which is the right most]. Do you see all these sensors physically available?

(2) Next possibility is that RGB sensor is physically present but it is non functional. I am not sure what can be the reason, but may be some loose connection inside the casing, or not enough power? To debug any such problem, I think only Intel can help.

(3) Something specifically incompatible with the Hardware where you are connecting the camera, thereby not identifying the RGB sensor. However, if you have cross checked all patches etc, then this is less likely to be case.

I am not sure what you could try next, but it may be worth trying to see how camera works on any other system.

(1 If you have access to Win 10 PC, then you should probably see how realsense-viewer is working.

(2) Likewise if you can get access to an Ubuntu PC, just check if realsense-viewer still detects only Stereo camera.

I can understand the tricky situation you are in. I hope that some one from Intel will be able to help you out

[Ncare]

update: Intel support have told me that the D435 doesn't have RGB.

To me this sounds like nonsense? I can clearly see an RGB camera/sensor on the front of the camera, and from the forums it seems that other people seem to have received colour streams from their D435. The specifications page clearly lists D4 vision processor with RGB resolution and FoV.

Main Intel® RealSense™ component

Intel® RealSense™ Vision Processor D4

Intel® RealSense™ module D430

Depth Field of View (FOV)—(Horizontal × Vertical × Diagonal)91.2 x 65.5 x 100.6 (+/- 3°)Depth Stream Output ResolutionUp to 1280 x 720Depth Stream Output Frame RateUp to 90 fpsMinimum Depth Distance (Min-Z)0.2mSensor Shutter TypeGlobal shutterMaximum RangeApprox.10 meters; Varies depending on calibration, scene, and lighting conditionRGB Sensor Resolution and Frame Rate1920 x 1080 at 30 fpsRGB Sensor FOV (Horizontal x Vertical x Diagonal)69.4° x 42.5° x 77° (+/- 3°)

Can anyone shed some light on this? Hopefully it's a communication issue.

@PS: I'm using Ubuntu. I've tried plugging the realsense into both windows and OSX, none of them recognise an RGB camera. The RGB is visible externally, as noted. Hardware/power/loose connection is looking likely, unless Intel is right about D435 not having RGB at all.

[MartyG]

The D435 does have an RGB sensor. It is the circuit board inside (called the D430 Module) that does not have an RGB sensor integrated into it. So the RGB sensor is connected to the cased USB D435 camera as a separate component rather than being an integrated part of the circuit board.

[Ncare]

That was my understanding!

Intel customer support have told me the following:

The D435 camera does not have an active RGB module. This information is stated on the following link:

https://realsense.intel.com/stereo/ https://realsense.intel.com/stereo/

Click on "Compare Depth Modules" to see a comparison table of features. It shows that the D415 is the only depth module with an RGB sensor.

I think they must not have understood my problem. Hopefully they will get back to me.

[PSnip]

Yes, they might have not understood you correctly.

I have a D435 camera and RGB stream works just fine.

As Marty mentioned, RGB sensor is connected to Asic Board via a connector. In case connector is loose inside, you may see this problem.

[Edward_T_Intel]

Odd choice for nomenclature and numbering; modules are only the imager boards and cameras are modules plus processor. The depth module, D430 module does not have an RGB camera; the D415 module has an integrated RGB camera. The D435 camera contains a D430 module AND a standalone RGB camera. The D4 processor board determines its mode by what is connected and it appears that there is something wrong with your RGB module. As someone else suggested, it might be that the flex cable is not firmly connected; perhaps the RGB camera itself is bad.

In any case the D4 thinks it is a D430 camera because your earlier information indicated a PID of 0x0ad4; a D435 camera (D430+RGB) has a PID 0x0B07.

[NWeib]

Hi NCare

I have a similar issue (I think). I purchase 4 Intel Real Sense D435, they all came in a box saying D435, and they have a sticker on them that say D435, however, one is recognized as D435 and show dept, infrared and RGB images when connected, while the other 3 are recognized as D430 and only show Infrared sensor, no RGB.

Have you been able to find a solution in the meantime?

I am planning to send the three RealSense that don't work back and purchase 3 new, but I am worried that the same will happen...

Let me know and thanks

NW

[MartyG]

I recall one case where the user opened the camera's casing and found that the RGB sensor's cable was disconnected, like the users above suggested might be the cause. They reconnected the cable and said that the camera was recognized as a D435. You should not do this if you plan to return them though, as opening them up would invalidate the warranty.

I would recommend exploring other possibilities before you return them. To begin with, you could unplug the USB cable from the one camera that works and put it on one of the other cameras (the cable unplugs from the side of the camera). Then try it in the same USB port that you normally use the working camera in. This will eliminate the possibility of a bad batch of USB-C cables. Problems are especially liable to occur if using your own choice of USB-C cable instead of the short one supplied in the box, as USB-C cables can have quality issues.

[AWit]

@Ncare​ @NWeib​ 

I believe I just had a very similar issue and found a solution for my case. Posting here in case it helps, even though I did not read the whole thread.

In short: I downgraded the camera firmware to a previous version and it worked. After re-upgrading to the current version, it continued to work.

DETAILS: I was trying to debug a separate issue with my D435s on Windows 10 (I'm suddenly getting lots of frame drops, when recording with Realsense Viewer, using even the most basic settings). I uninstalled the SDK and then reinstalled (v2.17.0). That didn't solve my problem, but the two cameras that I am using DID register as D435 and DID show the color stream. So tried upgrading the firmware on the 2 cameras to v5_11_1. That went smoothly, but when I started Realsense Viewer, one of the cameras (ONLY one) showed no RGB module. I tried restarting Viewer and moving the camera to different ports. I tried resetting the device. I ran the upgrade to v5_11_1 again for that camera, being very careful to ensure that it worked. After several iterations, none of this worked. So I tried downgrading to firmware v5_10_6. The camera then showed up as a D435, as expected, in Realsense Viewer. I upgraded to v5_11_1 for a third time. This time, it stuck, and the camera works (so far). So, the key was to downgrade the firmware and then upgrade again.

[ASper5]

Hello

Im having issues related to RGB Stream with the 435 and it seems to be related to firmware.

Actually, I have two cameras: a 435 and a 435i.

When I upload the most recent firmware to the 435i (Signed_Image_UVC_5_11_1_100.bin) it runs smoothly, but when I do the same with the 435 it gives this warning in the viewer:

When using 5_10_6_0 OR 5_10_3_0 and trying to access RGB:

It seems that it is giving this message because the viewer is requesting info that is missing as the RGB stream doesnt work.

Thanks

[ASper5]

Just realized image pasting doesnt work (even if they were shown on editor)

The first error was related to RGB parameters error (I dont have the image anymore)

The second was related to failed flush.

[ASper5]

Ive restarted the OS and the 435 is working with 5_10_6_0

What I find curious is that inside device manager (windows) 435i counts as "Camera", and the 435 counts as "Imaging Device".