Custom/DIY Stereo Image Sensor module with RealSense Vision Processor D4 Board

[AYuen1]

Hi all,

I'm fairly new to the RealSense development community. In fact, my company (www.navatics.com) has just bought some Vision Processor D4 boards and D430 Depth modules for R&D. My team has some experience working with image processors, and are looking to make a stereo camera to put on our future robotics products.

I have the following questions:

A) Is it possible and/or feasible to make a custom depth module (out of 2x 2-lane MIPI-CSI2 image sensors)?

I understand that the D4 board enumerates differently according to the depth module attached. This means the D4 board somehow identifies the module (I'm guessing via reading image sensor I2C identification). Assuming a custom depth module is at least possible, I would like to also be able to:

B) develop applications on the D4 using the RealSense SDK 2.0 and enumerate with custom values. Any ideas?

C) Is the tracking module on the D4 card optional? In other words, can I connect only a depth module and not connect anything to the tracking module and still obtain a stereo depth image?

Thank you all in advance.

Sincerely,

Adrian Yuen

[MartyG]

Hi Adrian! Welcome to the community.

We had an inquiry from another user about MIPI and custom camera sensors recently. This picture-illustrated discussion may be helpful to you.

Page 40 onwards of the data sheet document for the 400 Series modules describes the workings of the D4 Vision Processor.

https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/Intel-RealSense-D400-Series-Datasheet.pdf https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/Intel-RealSen…

I believe that to develop apps for the D4 module using the RealSense SDK 2.0, you would need to combine the D4 with one of the Depth Modules in the 400 Series, as described in the data sheet above.

The T150 Tracking Module used with the D4 Vision Processor is a separate component that can be attached to the D4 via a 50-pin connector.

[AYuen1]

Thank you for the response, MartyG.

I have studied the document for the D400 series (that you linked) quite extensively. However, it doesn't tell the difference between Intel stereo depth modules and two 2-lane MIPI CSI2 cameras (disregarding the IR-related components).

Referring to page 43 of the aforementioned D400 series document, the depth module receptacle provides very straight forward connections for the MIPI cameras. Since it also says (in Table 3-10)that the RGB camera and IR modules on the stereo depth module is optional, it seems to me that it is possible to simply connect some OV2740 or OV9282 image sensors to the D4 board directly. What can I do to make the D4 board to accept other image sensors?

Regarding the tracking module, I understand that it is a separate module. You didn't answer my question, which is: can I not connect anything to the tracking module receptacle on the D4 board, only connect a stereo depth module, and still obtain stereo depth map output?

Adrian

[MartyG]

The details in the data sheet is the main reference source about the D4 vision Processor. I am not an employee of Intel and so do not have access to further materials about questions that are not answered by that document or are not covered by my knowledge, unfortunately. An Intel support team member, who can consult with the RealSense developer team, maybe able to answer your question better than I can.

The 50-pin connector on the D4 Vision Processor component is mainly designed for attaching the T150 Tracking Module and / or an RGB sensor. So you may be able to attach your own custom RGB sensor to the board instead of the one that Intel uses in the D435 camera.

[AYuen1]

Other than posting here, would you happen to know better ways to get in touch with the RealSense development team or Intel support team members?

[MartyG]

The Intel support team is the front-door for access to technical help. Members of product teams do sometimes decide to post here to offer help, but generally the Support Team members are the means by which questions are passed to developer teams if they cannot answer it themselves.

The Support Team hours are Monday to Friday, 9 am to 5 pm US Pacific time, I believe. One of them should respond to your question.

[AYuen1]

I see, thank you very much. Guess I'll have to wait for the support team.

[idata]

Hello ayuen,

All of the information necessary to integrate with the D4 ASIC is in the datasheet ( https://www.mouser.com/pdfdocs/Intel_D400_Series_Datasheet.pdf) and the CAD files ( https://www.intel.com/content/www/us/en/support/articles/000026841/emerging-technologies/intel-realsense-technology.html). There is no additional information available to build your own depth camera solution. Feel free to test and validate your own solution. Best Regards, Juan N.

[AYuen1]

Hi Juan N.

Thank you for your response.

Would it be possible for our companies to enter into an NDA, or for my company to join a developer program of some sort to obtain some more in-depth technical information?

E.g. reference D4 processor application schematics, reference stereo depth module design

Adrian

[MartyG]

Intel does have an NDA process that you can apply for. Intel support person Jesus G said in 2017 in response to a query about 'pro' level of support:

"There is a "pro" level of support available to customers who have an NDA and special business relationship with Intel. We can provide customers info on how to sign up and get them started on the NDA process. They can use the forum for public comms or http://customercare.intel.com for private comms."

So an Intel support member such as Juan should be able to get you started on that NDA registration process if you've indicated here that you would like to do so. Or you can use a private support ticket to initiate the discussion.

[idata]

Hello ayuen,

In order for to file an NDA for your company, you will need to contact a local Intel representative. This is a procedure that is out of the scope of the support forum. Regarding a more in depth technical information, the information in the datasheet and the CAD files is the most in depth information Intel provides to customers of any level, even with an NDA you would get the same information. The information in the datasheet and CAD files should be enough for you to create your own depth module solution. Best Regards, Juan N.