- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I tried to run example model ncappzoo/caffe/SSD_MobileNet, finally I got 79.89ms inference time and 456.47 MB/sec bandwidth from mvNCProfile.
I have a question about this inference time.
If I simply insert some codes in 'run.py' to check the processing time as below, it returns average 150~160ms.
import time
..
tic = time.time()
run_inference(infer_image, graph)
toc = time.time()
print("Inference time %.3f\n" %((toc - tic)*1000))
Why did I get different results using time.tme()?
I want to know the main different between this code and mvNCProfile.
Thanks.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@hra.kim The times listed by mvNCProfile is strictly the amount of time spent processing on the chip. It does not account for any overhead like USB transfer time or any pre-processing like image resizing or fp32 to fp16 conversions i.e. in the run.py sample code for SSD MobileNet. The USB overhead will vary from network to network as some networks require more data and some return more data.
If you are interested, you can calculate the average inference using the benchmarkncs app in the app zoo (https://github.com/movidius/ncappzoo/tree/master/apps/benchmarkncs). You can do that by using the following command in the benchmark_ncs directory: ./benchmarkncs.py ../../caffe/SSD_MobileNet ../../data/images 300 300
.
This should run inference on the images in the data/images directory and output a frames per second (fps) value (or the number inferences processed by the device per second). You should be able to calculate the average inference time for the images by dividing 1 second by the fps value you get from benchmarkncs.py. Let me know what average inference times you see. Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Tome_at_Intel Thank you for your help, I got 7.87fps@SSD and 5.24fps@TInyYolo :)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@hra.kim For the highest performance, make sure your graph file is compiled with the -s 12 option. What kind of system are you running this on? Thanks.
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page