frame

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Mobilenet SSD conversion error from Keras to Tensorflow to Movidius

Hello, I have a custom trained keras mobilenet model which i have converted to a tensorflow model successfully. I am now trying to convert it to movidius but I am running into this problem:

robert@atreus:~$ mvNCCheck -in input_1 -on predictions/concat dev/converter/TF_Model/tf_model.meta 
/usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/Convolution.py:47: SyntaxWarning: assertion is always true, perhaps remove parentheses?
  assert(False, "Layer type not supported by Convolution: " + obj.type)
mvNCCheck v02.00, Copyright @ Intel Corporation 2017

****** Info: No Weights provided. inferred path: dev/converter/TF_Model/tf_model.data-00000-of-00001******
dev/converter/TF_Model/tf_model.meta
2019-02-27 11:27:46.187790: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-02-27 11:27:46.209947: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz
2019-02-27 11:27:46.210526: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x4a0f550 executing computations on platform Host. Devices:
2019-02-27 11:27:46.210541: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
WARNING:tensorflow:From /home/robert/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Traceback (most recent call last):
  File "/usr/local/bin/mvNCCheck", line 239, in <module>
    quit_code = check_net(args.network, args.image, args.inputnode, args.outputnode, args.nshaves, args.inputsize, args.weights, args)
  File "/usr/local/bin/mvNCCheck", line 206, in check_net
    load_ret = load_network(args, parser, myriad_config)
  File "/usr/local/bin/ncsdk/Controllers/Scheduler.py", line 72, in load_network
    parsedLayers = p.parse(arguments)
  File "/usr/local/bin/ncsdk/Controllers/Parsers/TensorFlow.py", line 673, in parse
    parsed_op = opParser(op, operations, dummy_feed_dict)
  File "/usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/FusedBatchNorm.py", line 36, in load
    scale = np.reciprocal(np.sqrt(variance)) * scale_param
ValueError: operands could not be broadcast together with shapes (0,) (32,)

This is the original keras model that I am using to train:

https://github.com/tanakataiki/ssd_kerasV2/blob/master/model/ssd300MobileNet.py

After training I am converting to a tensorflow graph like this:


input_shape = (300, 300, 3) mobilenet_model = SSD(input_shape, num_classes=1) mobilenet_model.load_weights('mobilenet_test/weights.32-3.41.hdf5', by_name=True) def keras_to_tf( ): tf_model_path = './TF_Model/tf_model' saver = tf.train.Saver() with K.get_session() as sess: K.set_learning_phase(0) saver.save(sess, tf_model_path) return True keras_to_tf( )

Comments

  • 6 Comments sorted by Votes Date Added
  • Apparently there's a config file that needs to be used:
    https://ncsforum.movidius.com/discussion/1588/ncsdk-2-10-option-tf-ssd-config#latest

    after doing so, I get input/output node name errors even after converting from .meta to .pb (freezing the graph)

    @Jesus_at_Intel could it be that my model structure is wrong?
    https://github.com/tanakataiki/ssd_kerasV2/blob/master/model/ssd300MobileNet.py

  • I am stuck on this issue still, I am trying multiple ways of converting mobilenet to movidius and with no luck

    Here are some examples.

    robert@atreus:~/dev/converter$ mvNCCompile mobilenet_test/groceries/model.ckpt.meta  --tf-ssd-config tf_ssd.conf
    /usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/Convolution.py:47: SyntaxWarning: assertion is always true, perhaps remove parentheses?
      assert(False, "Layer type not supported by Convolution: " + obj.type)
    mvNCCompile v02.00, Copyright @ Intel Corporation 2017
    
    ****** Info: No Weights provided. inferred path: mobilenet_test/groceries/model.ckpt.data-00000-of-00001******
    mobilenet_test/groceries/model.ckpt.meta
    2019-02-27 16:18:54.843088: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2019-02-27 16:18:54.865891: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz
    2019-02-27 16:18:54.866276: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x518eb40 executing computations on platform Host. Devices:
    2019-02-27 16:18:54.866307: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
    WARNING:tensorflow:From /home/robert/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
    Instructions for updating:
    Use standard file APIs to check for files with this prefix.
    [Error 5] Toolkit Error: Stage Details Not Supported: FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_0/BatchNorm/batchnorm/mul_1
    
    
    robert@atreus:~/dev/converter$ mvNCCompile mobilenet_test/output_graph.pb  -in DecodeJpeg -on final_result
    /usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/Convolution.py:47: SyntaxWarning: assertion is always true, perhaps remove parentheses?
      assert(False, "Layer type not supported by Convolution: " + obj.type)
    mvNCCompile v02.00, Copyright @ Intel Corporation 2017
    
    mobilenet_test/output_graph.pb
    2019-02-27 16:37:54.877150: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2019-02-27 16:37:54.897864: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz
    2019-02-27 16:37:54.898292: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x631fef0 executing computations on platform Host. Devices:
    2019-02-27 16:37:54.898307: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
    2019-02-27 16:37:55.078046: W tensorflow/core/framework/op_def_util.cc:355] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().
    [Error 5] Toolkit Error: Stage Details Not Supported: Dynamic input shape not supported.
    
    

    These graphs were trained with the Object Detection methods Tensorflow provides on their Models/Research libraries

  • input_1
    conv1_pad
    conv1
    conv1_bn
    conv1_relu
    conv_pad_1
    conv_dw_1
    conv_dw_1_bn
    conv_dw_1_relu
    conv_pw_1
    conv_pw_1_bn
    conv_pw_1_relu
    conv_pad_2
    conv_dw_2
    conv_dw_2_bn
    conv_dw_2_relu
    conv_pw_2
    conv_pw_2_bn
    conv_pw_2_relu
    conv_pad_3
    conv_dw_3
    conv_dw_3_bn
    conv_dw_3_relu
    conv_pw_3
    conv_pw_3_bn
    conv_pw_3_relu
    conv_pad_4
    conv_dw_4
    conv_dw_4_bn
    conv_dw_4_relu
    conv_pw_4
    conv_pw_4_bn
    conv_pw_4_relu
    conv_pad_5
    conv_dw_5
    conv_dw_5_bn
    conv_dw_5_relu
    conv_pw_5
    conv_pw_5_bn
    conv_pw_5_relu
    conv_pad_6
    conv_dw_6
    conv_dw_6_bn
    conv_dw_6_relu
    conv_pw_6
    conv_pw_6_bn
    conv_pw_6_relu
    conv_pad_7
    conv_dw_7
    conv_dw_7_bn
    conv_dw_7_relu
    conv_pw_7
    conv_pw_7_bn
    conv_pw_7_relu
    conv_pad_8
    conv_dw_8
    conv_dw_8_bn
    conv_dw_8_relu
    conv_pw_8
    conv_pw_8_bn
    conv_pw_8_relu
    conv_pad_9
    conv_dw_9
    conv_dw_9_bn
    conv_dw_9_relu
    conv_pw_9
    conv_pw_9_bn
    conv_pw_9_relu
    conv_pad_10
    conv_dw_10
    conv_dw_10_bn
    conv_dw_10_relu
    conv_pw_10
    conv_pw_10_bn
    conv_pw_10_relu
    conv_pad_11
    conv_dw_11
    conv_dw_11_bn
    conv_dw_11_relu
    conv__11
    conv_11_bn
    conv_11_relu
    conv_pad_12
    conv_dw_12
    conv_dw_12_bn
    conv_dw_12_relu
    conv_pw_12
    conv_pw_12_bn
    conv_pw_12_relu
    conv_pad_13
    conv_dw_13
    conv_dw_13_bn
    conv_dw_13_relu
    conv__13
    conv_13_bn
    conv_13_relu
    conv_pad_14_1
    conv__14_1
    conv_14_bn_1
    conv_14_relu_1
    conv__14_2
    conv_14_bn_2
    conv_14_relu_2
    conv_pad_15_1
    conv__15_1
    conv_15_bn_1
    conv_15_relu_1
    conv__15_2
    conv_15_bn_2
    conv_15_relu_2
    conv_pad_16_1
    conv__16_1
    conv_16_bn_1
    conv_16_relu_1
    conv__16_2
    conv_16_bn_2
    conv_16_relu_2
    conv_pad_17_1
    conv__17_1
    conv_17_bn_1
    conv_17_relu_1
    conv__17_2
    conv11_mbox_conf
    conv13_mbox_conf
    conv14_2_mbox_conf
    conv15_2_mbox_conf
    conv16_2_mbox_conf
    conv17_2_mbox_conf
    conv11_mbox_loc
    conv13_mbox_loc
    conv14_2_mbox_loc
    conv15_2_mbox_loc
    conv16_2_mbox_loc
    conv17_2_mbox_loc
    conv11_mbox_conf_flat
    conv13_mbox_conf_flat
    conv14_2_mbox_conf_flat
    conv15_2_mbox_conf_flat
    conv16_2_mbox_conf_flat
    conv17_2_mbox_conf_flat
    conv11_mbox_loc_flat
    conv13_mbox_loc_flat
    conv14_2_mbox_loc_flat
    conv15_2_mbox_loc_flat
    conv16_2_mbox_loc_flat
    conv17_2_mbox_loc_flat
    mbox_conf
    mbox_loc
    mbox_conf_logits
    conv11_mbox_priorbox
    conv13_mbox_priorbox
    conv14_2_mbox_priorbox
    conv15_2_mbox_priorbox
    conv16_2_mbox_priorbox
    conv17_2_mbox_priorbox
    mbox_loc_final
    mbox_conf_final
    mbox_priorbox
    predictions
    

    This is my model structure, I'm not sure if there's any layers that are not compatible with mvNCCompile and if so, how to fix it.

  • Hi @robert_osmosis

    Thanks so much for your patience.

    There are sometimes issues when converting custom models trained in Keras to Tensorflow to NCS Graph. And the "stage details details not supported:" errors means that there are layers or operations that you're using that are still unsupported by the NCSDK. By just looking at these error messages, that's all I'm able to tell. Can you share your model and procedures with us? I want to try to convert your model on my side and I'll share the results with you.

    Also, try to enable the debug option in the Tensorflow parser, this could provide us additional information about the error. You will need to set the debug flag to True in the following file as sudo:
    /usr/local/bin/ncsdk/Controllers/Parsers/TensorFlow.py

    Sincerely,
    Sahira

  • Thank you for the reply @Sahira_at_Intel

    https://drive.google.com/drive/folders/1wXGvPXLqf5iPrHk-gozKDyu9CWrPErHH?usp=sharing

    Here is the model, which I converted doing this:

    with K.get_session() as sess:
            K.set_learning_phase(0)
            print("wat")
            saver.save(sess, tf_model_path)
    
            frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["predictions/concat"])
            graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False)
    
    

    After setting the debug flag to True, this is the output

    robert@atreus:~/dev/converter$ mvNCCompile inference_graph.pb  --tf-ssd-config tf_ssd.conf  
    /usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/Convolution.py:47: SyntaxWarning: assertion is always true, perhaps remove parentheses?
      assert(False, "Layer type not supported by Convolution: " + obj.type)
    mvNCCompile v02.00, Copyright @ Intel Corporation 2017
    
    inference_graph.pb
    2019-03-07 13:44:57.777974: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
    2019-03-07 13:44:57.799708: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz
    2019-03-07 13:44:57.800257: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x6049030 executing computations on platform Host. Devices:
    2019-03-07 13:44:57.800284: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
    [Error 13] Toolkit Error: Provided OutputNode/InputNode name does not exist or does not match with one contained in model file Provided: . Could not infer output node names. Default names:num_detections,detection_scores,detection_classes,detection_boxes
    
    
  • Hi @robert_osmosis

    Since NCSDK 2.10 is compatible with the SSD_mobilenet_v1/v2_coco models, you can try re-training a model based on SSD_mobilenet_v1/v2_coco with your data. Afterwards, you can try converting that newly retrained model to an NCS graph. The SSD_mobilenet_v1/v2_coco models can be found here . Please let me know if you were able to successfully convert the model + deploy.

    Best Regards,
    Sahira

Sign In or Register to comment.