cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
king32
Visitor
Visitor
268 Views
Registered: ‎12-04-2017

Question about yaml file of models in AI Model Zoo

Jump to solution

Hello.

I am trying to implement a neural network on ZCU102 MPSoC (in particular ResNet50 NN). As I followed the tutorial from Xilinx resources, I can download pre-trained models from AI Model Zoo, Quantize my model, Compile (Or cross-compile) model, then transfer the results to my board.

This is an example *.yaml file from Xilinx:


# Copyright 2019 Xilinx Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


description: resnetv1_50 classifier on ImageNet.
input size: 224*224
float ops: 6.97G
task: classification
framework: tensorflow
prune: 'no'
version: 1.3
files:
- name: tf_resnetv1_50_imagenet_224_224_6.97G_1.3
type: float & quantized
board: GPU
download link: https://www.xilinx.com/bin/public/openDownload?filename=tf_resnetv1_50_imagenet_224_224_6.97G_1.3.zip
checksum: a766ce47e318cd227fb046c677282da2
- name: resnet_v1_50_tf
type: xmodel
board: zcu102 & zcu104
download link: https://www.xilinx.com/bin/public/openDownload?filename=resnet_v1_50_tf-zcu102_zcu104-r1.3.1.tar.gz
checksum: c167dbc411a78476f76d3226e2dc718e
- name: resnet_v1_50_tf
type: xmodel
board: vck190
download link: https://www.xilinx.com/bin/public/openDownload?filename=resnet_v1_50_tf-vck190-r1.3.1.tar.gz
checksum: c657984006849aaf2a573639e6b2a92c
- name: resnet_v1_50_tf
type: xmodel
board: u50
download link: https://www.xilinx.com/bin/public/openDownload?filename=resnet_v1_50_tf-u50-r1.3.1.tar.gz
checksum: af527bba726bbd07065905a223068d86
- name: resnet_v1_50_tf
type: xmodel
board: u50lv9e & u50lv10e & u280
download link: https://www.xilinx.com/bin/public/openDownload?filename=resnet_v1_50_tf-u50lv-u280-r1.3.1.tar.gz
checksum: c2fa71c6baa857c86d09d55e0c009fa7
- name: resnet_v1_50_tf
type: xmodel
board: u50-v3me & u50lv-v3me & u280-v3me
download link: https://www.xilinx.com/bin/public/openDownload?filename=resnet_v1_50_tf-u50-u50lv-u280-v3me-r1.3.1.tar.gz
checksum: bac36659566cb3d884a09d8384351d3b
- name: resnet_v1_50_tf
type: xmodel
board: u200 & u250
download link: https://www.xilinx.com/bin/public/openDownload?filename=resnet_v1_50_tf-u200-u250-r1.3.0.tar.gz
checksum: 57162bfcfdd5b91b4b0fee7e62124a1b
license: https://github.com/Xilinx/Vitis-AI/blob/master/LICENSE

 

I tried to download board: GPU and then take steps to Quantize the model, but unfortunately, since my graphic card is not that powerful (4GB DDR5), I get OOM error (Out of Memory) and I cannot continue with the project. Now I have two questions:

1- What is the "type: xmodel" of the file provided in this *.yaml file? Is this something pre-quantized and pre-compiled and all I need is to copy it to the board? How should I use it? Because I found no tutorial for using this "type: xmodel" for implementing the NN. All of the tutorials take GPU type, quantize the mode, then cross-compile it and done. However, as I said I don't have a powerful GPU.

1- Since I am not an expert in AI and machine learning and I am more of a Hardware designer guy, are people really Quantize and Compile the model in their laptop computer? Or maybe using something like data lake for this?

 

Thank you for your help

 

0 Kudos
1 Solution

Accepted Solutions
robinplc2
Participant
Participant
137 Views
Registered: ‎06-11-2020

Hi again,

Actually, I have already posted an answer and I wonder why it's not here. 

Question: I am trying to make some research on the hardware structure of Xilinx DPU and I designed my own hardware scheme in Xilinx Vivado using DPU IP. As far as I understand, if I want to use my own configured DPU project and not the pre-designed hardware (which is included in the system image for ZCU102 in your provided link), I need to quantize and compile the model from scratch and I cannot use the xmodel type provided in that *.yaml file. I think this is also mentioned in DPU-TRD Vivado Flow (https://github.com/Xilinx/Vitis-AI/blob/master/dsa/DPU-TRD/prj/Vivado/README.mdPlease correct me if I am wrong.

Answer: Yes you are right, you need to recompile the quantized model against your new target architecture (Hope you are aware of the dpu architecture "json file", use against the Vitis ai compiler)

Question: If this is not the case and I can use the *.xmodel file (which as you said already compiled for specific targets) with any design including DPU cores, then I can make sure I don't need more effort for debugging and understanding the Neural Network and how to make it ready for ZCU102 board.

Answer: I think in your case, you don't want to make any changes with the Neural network. if that's the case, you can use the pre-quantized model from the model zoo. Then you don't need to have through the pain of having a GPU for training, quantize(finetuning). Just compile against your new hardware target, done

Question: By the way, do you think I need to use a more powerful GPU for preparing the ResNet50 for ZCU102? (My GPU is GeForce GTX 1650 4GB); If the GPU is the source of my problem then I think I need to use a more powerful PC or a cloud service. Do you have any GPU suggestions? Is it possible to run Vitis-AI docker on cloud services? if yes, do you have any suggested cloud platforms? 

Answer: Like I said before, if you need to make any changes against the pre-trained model from the model zoo(suppose you need to classify a new class that is not there in the Imagenet dataset(assuming Resnet50 is trained on it)) or say any custom modifications, then you need to retrain the model against your new specs (not explaining as it is outside the scope of the question). Here you might need a GPU depending on the model specs. Personally, I don't have experience with Resnet50 training, so if you get out of memory error with your 4GB card, you might need to update. I use a gtx 1080 Ti(11 GB one) but you have new options nowadays RTX cards from nvidia. I prefer my own local setup, but if you want a quicker way around, just grad a cloud service and install Vitis ai, and just use it. I am not sure whether any cloud services offer Vitis ai as one of their services. Anyway, it's not that difficult.

Hope most of your doubts are cleared!!

Best

 

View solution in original post

3 Replies
robinplc2
Participant
Participant
249 Views
Registered: ‎06-11-2020

Hi,

I am not sure whether I have understood your question correctly,

Since we are talking about TensorFlow models,

Question 1: "xmodel" are the ones you can directly copy to the board and use for your application. So basically they are already compiled for specific targets. Please refer to VART or Vitis AI library samples demo to know how to use these files, please refer https://github.com/Xilinx/Vitis-AI/blob/master/tools/Vitis-AI-Library/README.md#running-vitis-ai-library-examples.

 

Question 2: Quantization and compilation of the quantized model are generally done on your host PC/laptop.

Please let me know if you need more clarifications. Happy to help

 

king32
Visitor
Visitor
228 Views
Registered: ‎12-04-2017

Hi robin,

Thank you very much for your answer. I am trying to make some research on the hardware structure of Xilinx DPU and I designed my own hardware scheme in Xilinx Vivado using DPU IP. As far as I understand, if I want to use my own configured DPU project and not the pre-designed hardware (which is included in the system image for ZCU102 in your provided link), I need to quantize and compile the model from scratch and I cannot use the xmodel type provided in that *.yaml file. I think this is also mentioned in DPU-TRD Vivado Flow (https://github.com/Xilinx/Vitis-AI/blob/master/dsa/DPU-TRD/prj/Vivado/README.md) Please correct me if I am wrong.

If this is not the case and I can use the *.xmodel file (which as you said already compiled for specific targets) with any design including DPU cores, then I can make sure I don't need more effort for debugging and understanding the Neural Network and how to make it ready for ZCU102 board.

By the way, do you think I need to use a more powerful GPU for preparing the ResNet50 for ZCU102? (My GPU is GeForce GTX 1650 4GB); If the GPU is the source of my problem then I think I need to use a more powerful PC or a cloud service. Do you have any GPU suggestions? Is it possible to run Vitis-AI docker on cloud services? if yes, do you have any suggested cloud platforms? 

Once again thank you very much for your time.

0 Kudos
robinplc2
Participant
Participant
138 Views
Registered: ‎06-11-2020

Hi again,

Actually, I have already posted an answer and I wonder why it's not here. 

Question: I am trying to make some research on the hardware structure of Xilinx DPU and I designed my own hardware scheme in Xilinx Vivado using DPU IP. As far as I understand, if I want to use my own configured DPU project and not the pre-designed hardware (which is included in the system image for ZCU102 in your provided link), I need to quantize and compile the model from scratch and I cannot use the xmodel type provided in that *.yaml file. I think this is also mentioned in DPU-TRD Vivado Flow (https://github.com/Xilinx/Vitis-AI/blob/master/dsa/DPU-TRD/prj/Vivado/README.mdPlease correct me if I am wrong.

Answer: Yes you are right, you need to recompile the quantized model against your new target architecture (Hope you are aware of the dpu architecture "json file", use against the Vitis ai compiler)

Question: If this is not the case and I can use the *.xmodel file (which as you said already compiled for specific targets) with any design including DPU cores, then I can make sure I don't need more effort for debugging and understanding the Neural Network and how to make it ready for ZCU102 board.

Answer: I think in your case, you don't want to make any changes with the Neural network. if that's the case, you can use the pre-quantized model from the model zoo. Then you don't need to have through the pain of having a GPU for training, quantize(finetuning). Just compile against your new hardware target, done

Question: By the way, do you think I need to use a more powerful GPU for preparing the ResNet50 for ZCU102? (My GPU is GeForce GTX 1650 4GB); If the GPU is the source of my problem then I think I need to use a more powerful PC or a cloud service. Do you have any GPU suggestions? Is it possible to run Vitis-AI docker on cloud services? if yes, do you have any suggested cloud platforms? 

Answer: Like I said before, if you need to make any changes against the pre-trained model from the model zoo(suppose you need to classify a new class that is not there in the Imagenet dataset(assuming Resnet50 is trained on it)) or say any custom modifications, then you need to retrain the model against your new specs (not explaining as it is outside the scope of the question). Here you might need a GPU depending on the model specs. Personally, I don't have experience with Resnet50 training, so if you get out of memory error with your 4GB card, you might need to update. I use a gtx 1080 Ti(11 GB one) but you have new options nowadays RTX cards from nvidia. I prefer my own local setup, but if you want a quicker way around, just grad a cloud service and install Vitis ai, and just use it. I am not sure whether any cloud services offer Vitis ai as one of their services. Anyway, it's not that difficult.

Hope most of your doubts are cleared!!

Best

 

View solution in original post