I ran through something interesting while trying to quantize a custom onnx model using the
quantization only works for input shape of
(1, ., .)
How did I end up there
I am currently doing a project where i customize the resnet18 model, so that i delete everything below the last relu and add a new output right after it.
At this point my custom resnet takes as input
tensor: float32[N,3,224,224] and outputs
Then, in python, I run the following code:
# load onnx & update opset to 14 model = load_model(RESNET18_V1_WOGAP_FILE) model = version_converter.convert_version(model, 14) # quantize model # I tried many possibilities for calibration_input_set (numpy, tensor, different dimensions) calibration_input_set = torch.FloatTensor(100, 3, 224, 224).uniform_(-100, 100) qmodel = compile_onnx_model( model, calibration_input_set, )
Then when I run the code I get the error:
ValueError: Got 1 inputs, expected 101. Either the quantized module has not been properly initialized or the input data has been changed since its initialization.
After a bit of research, i found that it was due to the call to
quantize_module, in the
build_quantize_module function (cf: source code):
# Build the quantized module # TODO: mismatch here. We traced with dummy_input_for_tracing which made some operator # only work over shape of (1, ., .). For example, some reshape have newshape hardcoded based # on the inputset we sent in the NumpyModule. quantized_module = post_training_quant.quantize_module(*inputset_as_numpy_tuple)
I have 2 questions stemming from this:
- What are the consequences of this? I couldn’t find any related issue.
- Is There a way to bypass this input shape restriction?
Thanks in advance,