Inception-resnet-v2 NaN

[idata]

I'm trying to get an inception-resnet-v2 network running on the movidius stick, but it gives me a NaN result.

I'm using caffe for this one, not tensorflow. Gotten from https://github.com/soeaver/caffe-model/blob/master/cls/inception/deploy_inception-resnet-v2.prototxt

I'm using it pretrained, and replaced the final fully connected layer to train it on our own data. All other weights are not changed during training.

I tried pinpointing where it goes wrong and the Eltwise layer 'inception_resnet_v2_a1_residual_eltwise' seems to be the first where the result of caffe and the result of ncs seem to differ significantly (180%, compared to 0.5% the layers before). Which eventually leads to consistent NaN outputs at the end of the network.

This is my prototxt:

https://gist.github.com/anonymous/1308671b77212b5397f5319c074336c6

This is the pretrained network:

https://drive.google.com/open?id=0B9mkjlmP0d7zNFY1b2s4NXVsRVU

[idata]

@brainshift, is Caffe a hard requirement, or can you use TensorFlow instead? If you are open to using TensorFlow, use https://github.com/movidius/ncappzoo/blob/master/tensorflow/inception/Makefile as a template to run inception-resnet-v2. Pretrained checkpoint files are available here - https://github.com/tensorflow/models/tree/master/research/slim#Pretrained. You will have to extract the output_node_name from tensorboard.

[idata]

I have the same problem with Eltwise layer in SFD face detector https://github.com/sfzhang15/SFD .

Results of mvNCCheck for full network:

Blob generatedUSB: Transferring Data…USB: Myriad Execution FinishedUSB: Myriad Connection Closing.USB: Myriad Connection Closed.Result: (1, 750, 7)1) 51 nan2) 72 nan3) 79 nan4) 23 nan5) 44 nanExpected: (1, 1, 7)1) 0 0.02) 6 -1.03) 5 -1.04) 4 -1.05) 3 -1.0/usr/local/bin/ncsdk/Controllers/Metrics.py:75: RuntimeWarning: invalid value encountered in greater------------------------------------------------------------ Obtained values------------------------------------------------------------ Obtained Min Pixel Accuracy: nan% (max allowed=2%), Fail Obtained Average Pixel Accuracy: nan% (max allowed=1%), Fail Obtained Percentage of wrong values: 64100.0% (max allowed=0%), Fail Obtained Pixel-wise L2 error: nan% (max allowed=1%), Fail Obtained Global Sum Difference: nan------------------------------------------------------------

When I remove branch contained Eltwise layer from network the Obtained values are Pass.