frame

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Inception-resnet-v2 NaN

I'm trying to get an inception-resnet-v2 network running on the movidius stick, but it gives me a NaN result.
I'm using caffe for this one, not tensorflow. Gotten from https://github.com/soeaver/caffe-model/blob/master/cls/inception/deploy_inception-resnet-v2.prototxt

I'm using it pretrained, and replaced the final fully connected layer to train it on our own data. All other weights are not changed during training.

I tried pinpointing where it goes wrong and the Eltwise layer 'inception_resnet_v2_a1_residual_eltwise' seems to be the first where the result of caffe and the result of ncs seem to differ significantly (180%, compared to 0.5% the layers before). Which eventually leads to consistent NaN outputs at the end of the network.

This is my prototxt:
https://gist.github.com/anonymous/1308671b77212b5397f5319c074336c6

This is the pretrained network:
https://drive.google.com/open?id=0B9mkjlmP0d7zNFY1b2s4NXVsRVU

Comments

  • 2 Comments sorted by Votes Date Added
  • @brainshift, is Caffe a hard requirement, or can you use TensorFlow instead? If you are open to using TensorFlow, use https://github.com/movidius/ncappzoo/blob/master/tensorflow/inception/Makefile as a template to run inception-resnet-v2. Pretrained checkpoint files are available here - https://github.com/tensorflow/models/tree/master/research/slim#Pretrained. You will have to extract the output_node_name from tensorboard.

  • I have the same problem with Eltwise layer in SFD face detector https://github.com/sfzhang15/SFD .

    Results of mvNCCheck for full network:
    Blob generated
    USB: Transferring Data...
    USB: Myriad Execution Finished
    USB: Myriad Connection Closing.
    USB: Myriad Connection Closed.
    Result: (1, 750, 7)
    1) 51 nan
    2) 72 nan
    3) 79 nan
    4) 23 nan
    5) 44 nan
    Expected: (1, 1, 7)
    1) 0 0.0
    2) 6 -1.0
    3) 5 -1.0
    4) 4 -1.0
    5) 3 -1.0
    /usr/local/bin/ncsdk/Controllers/Metrics.py:75: RuntimeWarning: invalid value encountered in greater
    ------------------------------------------------------------
    Obtained values
    ------------------------------------------------------------
    Obtained Min Pixel Accuracy: nan% (max allowed=2%), Fail
    Obtained Average Pixel Accuracy: nan% (max allowed=1%), Fail
    Obtained Percentage of wrong values: 64100.0% (max allowed=0%), Fail
    Obtained Pixel-wise L2 error: nan% (max allowed=1%), Fail
    Obtained Global Sum Difference: nan
    ------------------------------------------------------------

    When I remove branch contained Eltwise layer from network the Obtained values are Pass.

Sign In or Register to comment.