cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
ARaoofy
Observer
Observer
384 Views
Registered: ‎09-01-2020

Quantizing TensorFlow Hub Models (Transfer Learning Workflow)

Dear Xilinx team,

hi, I am working on a ResNet50V2-based model for a project which has a very common Transfer Learning-style in Keras. We want to deploy it on the ZCU102 board for benchmarking using Vitis AI. however, the Vitis AI workflow fails. Here are the details: In our model, we use a TensorFlow Hub module (v1 format) followed by a dense layer. However, the vai_q_tensorflow seems to fail when quantizing the model.

Here is how the model is built: 

import tensorflow_hub as hub
hub_layer = hub.KerasLayer("https://tfhub.dev/google/remote_sensing/bigearthnet-resnet50/1", trainable=False, input_shape=(input_height,input_width, input_chan))
model = tf.keras.Sequential()
model.add(hub_layer)
model.add(tf.keras.layers.Dense(num_classes, activation='sigmoid'))

After successful training (which goes successfully through on both the CPU and GPU containers) and freezing of the model, I run vai_q_tensorflow as following:

> vai_q_tensorflow quantize --input_frozen_graph ./build/chkpts/frozen_model.pb --output_dir ./build/quantize/ --input_nodes 'input_2' --output_nodes 'dense/Sigmoid' --input_shapes "?,224,224,3" --input_fn image_input_fn.calib_input

...
ValueError: Input 1 of node keras_layer/StatefulPartitionedCall was passed float from resnet/root_block/conv2d/kernel:0 incompatible with expected resource.

There seems to be a problem with the way the hub layer is represented (keras_layer/StatefulPartitionedCall) and treated with vai_q_tensorflow. The closest similar problem to this that I found in the forum is the following link, but the discussions there did not help: https://forums.xilinx.com/t5/AI-and-Vitis-AI/incompatible-with-expected-float/td-p/994439 

 

One additional note: by replacing the hub_layer in the model with a Keras application model, i.e., keras.applications.ResNet50, I am able to complete the full workflow, so I believe the issue is really in the hub models.

I very much appreciate your help here: is hub models the problem? is there a workaround and if not is there any plan to support them in Vitis AI?

Thanks!

0 Kudos
3 Replies
ARaoofy
Observer
Observer
243 Views
Registered: ‎09-01-2020

Is there any opinion on this issue? 

0 Kudos
dvukadin
Contributor
Contributor
202 Views
Registered: ‎07-19-2018

Hi @ARaoofy 

I tried hub models two or three years ago and it did not work with the Xilinx workflow (it was DNNDK / AI_SDK_ 2.0.5 toolchain). I recommend you to give up from hub models and try something else. You can try tensorflow slim models, it should work with Xilinx workflow.

Regards,

dvukadin

ARaoofy
Observer
Observer
194 Views
Registered: ‎09-01-2020

I already gave up on it, did not know about the slim models and I moved to a similar approaches https://github.com/qubvel/segmentation_models and https://github.com/qubvel/classification_models , but the repositories are a bit old and are not under Tensorflow and slim seems to be the better option here. Thanks for the suggestion! I will give it a try. 

 

Regards,

Amir

0 Kudos