Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

what is the different between mvNCProfile inference time and python time.time()?

idata
Employee
855 Views

Hi,

 

I tried to run example model ncappzoo/caffe/SSD_MobileNet, finally I got 79.89ms inference time and 456.47 MB/sec bandwidth from mvNCProfile.

 

I have a question about this inference time.

 

If I simply insert some codes in 'run.py' to check the processing time as below, it returns average 150~160ms.

 

import time .. tic = time.time() run_inference(infer_image, graph) toc = time.time() print("Inference time %.3f\n" %((toc - tic)*1000))

 

Why did I get different results using time.tme()?

 

I want to know the main different between this code and mvNCProfile.

 

Thanks.

0 Kudos
3 Replies
idata
Employee
521 Views

@hra.kim The times listed by mvNCProfile is strictly the amount of time spent processing on the chip. It does not account for any overhead like USB transfer time or any pre-processing like image resizing or fp32 to fp16 conversions i.e. in the run.py sample code for SSD MobileNet. The USB overhead will vary from network to network as some networks require more data and some return more data.

 

If you are interested, you can calculate the average inference using the benchmarkncs app in the app zoo (https://github.com/movidius/ncappzoo/tree/master/apps/benchmarkncs). You can do that by using the following command in the benchmark_ncs directory: ./benchmarkncs.py ../../caffe/SSD_MobileNet ../../data/images 300 300.

 

This should run inference on the images in the data/images directory and output a frames per second (fps) value (or the number inferences processed by the device per second). You should be able to calculate the average inference time for the images by dividing 1 second by the fps value you get from benchmarkncs.py. Let me know what average inference times you see. Thanks.

0 Kudos
idata
Employee
521 Views

@Tome_at_Intel Thank you for your help, I got 7.87fps@SSD and 5.24fps@TInyYolo :)

0 Kudos
idata
Employee
521 Views

@hra.kim For the highest performance, make sure your graph file is compiled with the -s 12 option. What kind of system are you running this on? Thanks.

0 Kudos
Reply