02-24-2021 03:16 PM - edited 02-24-2021 03:17 PM
Dear Xilinx team,
hi, I am working on a ResNet50V2-based model for a project which has a very common Transfer Learning-style in Keras. We want to deploy it on the ZCU102 board for benchmarking using Vitis AI. however, the Vitis AI workflow fails. Here are the details: In our model, we use a TensorFlow Hub module (v1 format) followed by a dense layer. However, the vai_q_tensorflow seems to fail when quantizing the model.
Here is how the model is built:
import tensorflow_hub as hub
hub_layer = hub.KerasLayer("https://tfhub.dev/google/remote_sensing/bigearthnet-resnet50/1", trainable=False, input_shape=(input_height,input_width, input_chan))
model = tf.keras.Sequential()
After successful training (which goes successfully through on both the CPU and GPU containers) and freezing of the model, I run vai_q_tensorflow as following:
> vai_q_tensorflow quantize --input_frozen_graph ./build/chkpts/frozen_model.pb --output_dir ./build/quantize/ --input_nodes 'input_2' --output_nodes 'dense/Sigmoid' --input_shapes "?,224,224,3" --input_fn image_input_fn.calib_input
ValueError: Input 1 of node keras_layer/StatefulPartitionedCall was passed float from resnet/root_block/conv2d/kernel:0 incompatible with expected resource.
There seems to be a problem with the way the hub layer is represented (keras_layer/StatefulPartitionedCall) and treated with vai_q_tensorflow. The closest similar problem to this that I found in the forum is the following link, but the discussions there did not help: https://forums.xilinx.com/t5/AI-and-Vitis-AI/incompatible-with-expected-float/td-p/994439
One additional note: by replacing the hub_layer in the model with a Keras application model, i.e., keras.applications.ResNet50, I am able to complete the full workflow, so I believe the issue is really in the hub models.
I very much appreciate your help here: is hub models the problem? is there a workaround and if not is there any plan to support them in Vitis AI?
05-07-2021 01:08 AM
I tried hub models two or three years ago and it did not work with the Xilinx workflow (it was DNNDK / AI_SDK_ 2.0.5 toolchain). I recommend you to give up from hub models and try something else. You can try tensorflow slim models, it should work with Xilinx workflow.
05-07-2021 01:33 AM
I already gave up on it, did not know about the slim models and I moved to a similar approaches https://github.com/qubvel/segmentation_models and https://github.com/qubvel/classification_models , but the repositories are a bit old and are not under Tensorflow and slim seems to be the better option here. Thanks for the suggestion! I will give it a try.