frame

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Alert: Beginning Tuesday, June 25th, we will be freezing this site and migrating the content and forums to our new home at https://forums.intel.com/s/topic/0TO0P000000PqZDWA0/intel-neural-compute-sticks. Check it out now!

Tensorflow Finetuned models GPU device error

Comments

  • 2 Comments sorted by Votes Date Added
  • Hi There,

    I am using the Tensorflow Slim to finetune Inception model on new dataset for running on NCS. On running compile, getting error for GPU device (see below). Is there any special requirements for NCS on how to train models. Can we use GPU for model training?

    mvNCCompile -s 12 flowers/model.ckpt-1000.meta -in input -on=InceptionV3/Predictions/Reshape_1
    mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

    /usr/local/lib/python3.5/dist-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead
    if d.decorator_argspec is not None), _inspect.getargspec(target))
    Traceback (most recent call last):
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1327, in _do_call
    return fn(*args)
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1297, in _run_fn
    self._extend_graph()
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1358, in _extend_graph
    self._session, graph_def.SerializeToString(), status)
    File "/usr/lib/python3.5/contextlib.py", line 66, in exit
    next(self.gen)
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
    tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation 'gradients/softmax_cross_entropy_loss/Mul_grad/BroadcastGradientArgs': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/cpu:0 ]. Make sure the device specification refers to a valid device.
    [[Node: gradients/softmax_cross_entropy_loss/Mul_grad/BroadcastGradientArgs = BroadcastGradientArgs[T=DT_INT32, _device="/device:GPU:0"](gradients/softmax_cross_entropy_loss/Mul_grad/Shape, gradients/softmax_cross_entropy_loss/Mul_grad/Shape_1)]]

  • @gchauhan For training, the training method probably isn't an issue whether it is CPU or GPU based. However, we have not tested the SDK with the GPU flavor of Tensorflow and I think this could be the problem here. Please try using the the CPU flavor of Tensorflow and see if this resolves the issue. Thanks!

Sign In or Register to comment.