Compiling quantized net

[idata]

Hey, i'm a student doing a project with the ncs trying to compare it to other solutions. Since it's architecture is vector based and it supports int8 operations it would get a great edge if i were to quantize the network.

I am only using pre-trained models and i was trying to use the auto-quantization through :https://www.tensorflow.org/versions/r1.3/performance/quantization.Tried it with inception v3 and inception_resnet_v2, but i cant seem to compile the quantized outputs.

tried to plain copy like thus:

bazel-bin/tensorflow/tools/graph_transforms/transform_graph \

--in_graph=inception_v3_frozen.pb \ --outputs="InceptionV3/Predictions/Reshape_1" --out_graph=/tmp/quantized_graph.pb \ --transforms='add_default_attributes strip_unused_nodes(type=float, shape="1,299,299,3") remove_nodes(op=Identity, op=CheckNumerics) fold_constants(ignore_errors=true) fold_batch_norms fold_old_batch_norms quantize_weights quantize_nodes strip_unused_nodes sort_by_execution_order'

threw :

[Error 5] Toolkit Error: Stage Details Not Supported: Max

so i tried to limit the optimizations as i thought some might not have been implemented:

bazel-bin/tensorflow/tools/graph_transforms/transform_graph \

--in_graph=inception_v3_frozen.pb \ --outputs="InceptionV3/Predictions/Reshape_1" --out_graph=quantized_graph2.pb \ --transforms=' strip_unused_nodes(type=float, shape="1,299,299,3") quantize_weights quantize_nodes'

threw:

Traceback (most recent call last):

File "/usr/local/bin/mvNCCompile", line 156, in create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights, args.explicit_concat, args.ma2480, args.scheduler, args) File "/usr/local/bin/mvNCCompile", line 137, in create_graph load_ret = load_network(args, parser, myriad_config) File "/usr/local/bin/ncsdk/Controllers/Scheduler.py", line 82, in load_network parse_ret = parse_tensor(arguments, myriad_conf) File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 315, in parse_tensor item_shape = output_item.shape.as_list() File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_shape.py", line 820, in as_list raise ValueError("as_list() is not defined on an unknown TensorShape.")ValueError: as_list() is not defined on an unknown TensorShape.

then i made an account to make this post but before i did i tried with another graph:

bazel-bin/tensorflow/tools/graph_transforms/transform_graph --in_graph=inception_resnet_v2_frozen.pb \

--inputs="input" \ --outputs="InceptionResnetV2/Logits/Predictions" --out_graph=../i_res_v2_qt.pb \ --transforms=' strip_unused_nodes(type=float, shape="1,299,299,3") quantize_weights quantize_nodes'

threw:

File "/usr/local/bin/mvNCCompile", line 156, in

create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights, args.explicit_concat, args.ma2480, args.scheduler, args) File "/usr/local/bin/mvNCCompile", line 137, in create_graph load_ret = load_network(args, parser, myriad_config) File "/usr/local/bin/ncsdk/Controllers/Scheduler.py", line 82, in load_network parse_ret = parse_tensor(arguments, myriad_conf) File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 225, in parse_tensor inputTensor = graph.get_tensor_by_name(inputnode + ':0') File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3654, in get_tensor_by_name return self.as_graph_element(name, allow_tensor=True, allow_operation=False) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3478, in as_graph_element return self._as_graph_element_locked(obj, allow_tensor, allow_operation) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3520, in _as_graph_element_locked "graph." % (repr(name), repr(op_name)))KeyError: "The name 'input:0' refers to a Tensor which does not exist. The operation, 'input', does not exist in the graph."thsi

now this seems to about the imput node but since i used the same compilation as the makefile and tried to preserve the input by the "--inputs="input" " option i dont know how to solve it.

While i'm at it i'd also like to try to use tinyolo if i can get this quantized and was wondering if anyone has any .pb models of it, tried to use the darkflow conversion but couldnt compile the output into a .graph either :neutral:

Anyhow sorry for the long post, let me know if any of you have encountered a similar issue.

Best regards, Nuno Magalhães.

[idata]

I'm encountering the same error.

as_list() is not defined on an unknown TensorShape.

Did you manage to solve it? I'd rather not retrain my network but I can't find a way to define the input node Tensorshape in the freeze_graph or transform_graph tools