frame

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

How to debug sdk infer wrong result?

Hi, I have a pre-trained inceptionv3.pb file, and convert to graph file successfully without error.

But the infer result is all NaN in python.

Debug log didn't help anything.

Start download to NCS...
D: [         0] ncFifoWriteElem:2592    No layout conversion  is needed 0

D: [         0] convertDataTypeAndLayout:170    src data type 1 dst data type 0

D: [         0] convertDataTypeAndLayout:172    SRC: w 299 h 299 c 3 w_s 12 h_s 3588 c_s 4

D: [         0] convertDataTypeAndLayout:174    DST: w 299 h 299 c 3 w_s 6 h_s 1794 c_s 2

D: [         0] ncFifoWriteElem:2617    write count 0 num_elements 2 userparam 0x7fb0fc143cd0

I: [         0] ncGraphQueueInference:3048  trigger start

I: [         0] ncGraphQueueInference:3145  trigger end

D: [         0] ncFifoReadElem:2686 No layout conversion  is needed 1

D: [         0] convertDataTypeAndLayout:170    src data type 0 dst data type 1

D: [         0] convertDataTypeAndLayout:172    SRC: w 1 h 1 c 4 w_s 8 h_s 8 c_s 2

D: [         0] convertDataTypeAndLayout:174    DST: w 1 h 1 c 4 w_s 16 h_s 16 c_s 4

D: [         0] ncFifoReadElem:2714 num_elements 2 userparam 0x7fb0fc143ef0 output length 16

Cost: 0.35880494117736816
output data is:
[ nan  nan  nan  nan]
*******************************************************************************
inception-v3 on NCS
*******************************************************************************
[3 2 1 0]
3 3:yumi nan
2 2:tudou nan
1 1:shulin nan
0 0:huangdi nan

I don't know where to start with, someone any suggestion?

Comments

  • 3 Comments sorted by Votes Date Added
  • I meet the same problem too

  • @chenxiaoyu3 Can you provide a link to your python code?

  • Hi,
    I get almost the same error, using the same model, but with different tensor dimensions once 32 x32 in the second version 299 x 299:
    - When I use the 32 x 32 model the prediction succeeds perfectly without errors.
    - When I use the 299 x 299 model instead I can not continue getting the following debug:

    D: [         0] ncDeviceCreate:307  ncDeviceCreate index 0
    
    D: [         0] ncDeviceCreate:307  ncDeviceCreate index 1
    
    1 neural compute devices found!
    D: [         0] ncDeviceCreate:307  ncDeviceCreate index 0
    
    D: [         0] ncDeviceCreate:307  ncDeviceCreate index 1
    
    D: [         0] ncDeviceOpen:523    File path /usr/local/lib/mvnc/MvNCAPI-ma2450.mvcmd
    
    I: [         0] ncDeviceOpen:529    ncDeviceOpen() XLinkBootRemote returned success 0
    
    I: [         0] ncDeviceOpen:567    XLinkConnect done - link Id 0
    
    D: [         0] ncDeviceOpen:581    done
    
    I: [         0] ncDeviceOpen:583    Booted 2-ma2450 -> VSC
    
    I: [         0] getDevAttributes:382    Device attributes
    
    I: [         0] getDevAttributes:385    Device FW version: 2.8.2450.16e
    
    I: [         0] getDevAttributes:387    mvTensorVersion 2.8 
    
    I: [         0] getDevAttributes:388    Maximum graphs: 10
    
    I: [         0] getDevAttributes:389    Maximum fifos: 20
    
    I: [         0] getDevAttributes:391    Maximum graph option class: 1
    
    I: [         0] getDevAttributes:393    Maximum device option class: 1
    
    I: [         0] getDevAttributes:394    Device memory capacity: 522059056
    
    Hello NCS! Device opened normally.
    I: [         0] ncGraphAllocate:959 Starting Graph allocation sequence
    
    I: [         0] ncGraphAllocate:1025    Sent graph
    D: [         0] ncGraphAllocate:1050    Graph Status 0 rc 0 
    
    D: [         0] ncGraphAllocate:1096    Input tensor w 299 h 299 c 3 n 1 totalSize 536406 wstide 6 hstride 1794 cstride 2 layout 0
    
    D: [         0] ncGraphAllocate:1109    output tensor w 1 h 1 c 2 n 1 totalSize 4 wstide 4 hstride 4 cstride 2 layout 1
    
    I: [         0] ncGraphAllocate:1163    Graph allocation completed successfully
    
    I: [         0] ncFifoCreate:2163   Init fifo
    I: [         0] ncFifoAllocate:2358 Creating fifo
    I: [         0] ncFifoCreate:2163   Init fifo
    I: [         0] ncFifoAllocate:2358 Creating fifo
    
    D: [         0] ncFifoWriteElem:2629    No layout conversion  is needed 0
    
    D: [         0] convertDataTypeAndLayout:169    src data type 1 dst data type 0
    
    D: [         0] convertDataTypeAndLayout:171    SRC: w 299 h 299 c 3 w_s 12 h_s 3588 c_s 4
    
    D: [         0] convertDataTypeAndLayout:173    DST: w 299 h 299 c 3 w_s 6 h_s 1794 c_s 2
    
    D: [         0] ncFifoWriteElem:2654    write count 0 num_elements 2 userparam 0x7f4fe3040d58
    
    I: [         0] ncGraphQueueInference:3089  trigger start
    
    I: [         0] ncGraphQueueInference:3186  trigger end
    
    E: [         0] dispatcherEventReceive:235  dispatcherEventReceive() Read failed -1
    
    E: [         0] eventReader:253 Failed to receive event, the device may have reset
    
    E: [         0] ncFifoReadElem:2735 Failed to read fifo element
    
Sign In or Register to comment.