取消
显示结果 
显示  仅  | 搜索替代 
您的意思是: 

Vitis-AI 1.3/TensorFlow2环境下使用自定义CNN模型进行Mnist分类

yolanda
Moderator
Moderator
0 0 148

BY Chao Zhang

注意:本论坛博客所有内容皆来源于Xilinx工程师,如需转载,请写明出处作者及赛灵思论坛链接并发邮件至cncrc@xilinx.com,未经Xilinx及著作权人许可,禁止用作商业用途 

 

Vitis-AI 1.3/TensorFlow2环境下使用自定义CNN模型进行Mnist分类

 

概要

 

本示例工程中我们会在TensorFlow2下使用Keras API创建一个自定义CNN网络,在Vitis-AI 1.3环境下编译成Xilinx DPU上运行的模型文件,并在Xilinx zynqMP上部署运行。

该示例工程应在Vitis-AI 1.3 Docker,vitis-ai-tensorflow2 conda环境中运行。

有关Mnist手写数字数据集,请参见http://yann.lecun.com/exdb/mnist/。

在该例程中我们会演示以下的步骤:

  • 使用TensorFlow2训练和评估小型自定义卷积神经网络
  • 对浮点模型进行量化(quantization)
  • 对量化模型进行评估
  • 使用校准数据集对训练后的模型进行微调(finetuning)
  • 编译量化和微调的模型,以创建.xmodel文件,准备在DPU加速器IP上执行
  • 在zynqMP评估板上下载并运行该模型

 

代码说明

 

可从下面的github页面获得源码。

https://github.com/lobster1989/Mnist-classification-Vitis-AI-1.3-TensorFlow2

 

Python 及Shell脚本源码说明如下:

load_data.py:下载Mnist数据集;

generate_images.py:从Keras在线数据集生成本地图像文件;

train.py:创建和训练用于Mnist分类的简单CNN模型, 经过训练的浮点模型将被保存到本地;

quantize.py:使用Vitis Quantizer量化前一个步骤保存的浮点模型,量化后的模型将被保存到本地;

eval_quantized.py:评估量化模型的accuracy

finetune.py:利用Vitis-AI进行模型微调(仅作演示,后续编译部署的为quantize步骤保存的模型);

compile_zcu102.sh/compile_zcu104.sh:启动vai_c_tensorflow2命令以将量化或微调的模型编译为供DPU运行的.xmodel文件。两个文件分别对应目标为zcu102zcu104开发板;

make_target_zcu102.sh/make_target_zcu104.sh:将.xmodel和图像复制到./target_zcu10x文件夹中,准备好将其复制到zcu102/zcu104评估板的SD卡中。

 

模型创建/量化/编译

 

在运行此部分之前,我们应在Vitis-AI docker环境下激活vitis-ai-tensorflow2 anaconda环境。 有关更多详细信息,请参阅最新版本的UG1414。

 

(vitis-ai-tensorflow2) Vitis-AI /workspace/myproj/tf2-mnist-end-to-end > python train.py

 

Load Mnist dataset..

 

Create custom cnn..

Model: "mnist_customcnn_model"

_________________________________________________________________

Layer (type)                 Output Shape              Param #  

=================================================================

input_1 (InputLayer)         [(None, 28, 28, 1)]       0        

_________________________________________________________________

conv2d (Conv2D)              (None, 26, 26, 32)        320      

_________________________________________________________________

max_pooling2d (MaxPooling2D) (None, 13, 13, 32)        0        

_________________________________________________________________

conv2d_1 (Conv2D)            (None, 11, 11, 64)        18496    

_________________________________________________________________

max_pooling2d_1 (MaxPooling2 (None, 5, 5, 64)          0        

_________________________________________________________________

conv2d_2 (Conv2D)            (None, 3, 3, 64)          36928    

_________________________________________________________________

flatten (Flatten)            (None, 576)               0        

_________________________________________________________________

dense (Dense)                (None, 64)                36928    

_________________________________________________________________

dense_1 (Dense)              (None, 10)                650      

=================================================================

Total params: 93,322

Trainable params: 93,322

Non-trainable params: 0

_________________________________________________________________

 

Fit on dataset..

Epoch 1/10

782/782 [==============================] - 13s 16ms/step - loss: 0.1843 - accuracy: 0.9427 - val_loss: 0.0701 - val_accuracy: 0.9813

Epoch 2/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0529 - accuracy: 0.9835 - val_loss: 0.0543 - val_accuracy: 0.9855

Epoch 3/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0346 - accuracy: 0.9894 - val_loss: 0.0472 - val_accuracy: 0.9877

Epoch 4/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0252 - accuracy: 0.9929 - val_loss: 0.0463 - val_accuracy: 0.9878

Epoch 5/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0188 - accuracy: 0.9945 - val_loss: 0.0494 - val_accuracy: 0.9896

Epoch 6/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0147 - accuracy: 0.9956 - val_loss: 0.0513 - val_accuracy: 0.9901

Epoch 7/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0121 - accuracy: 0.9966 - val_loss: 0.0452 - val_accuracy: 0.9920

Epoch 8/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0096 - accuracy: 0.9973 - val_loss: 0.0542 - val_accuracy: 0.9906

Epoch 9/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0088 - accuracy: 0.9976 - val_loss: 0.0640 - val_accuracy: 0.9893

Epoch 10/10

782/782 [==============================] - 5s 7ms/step - loss: 0.0073 - accuracy: 0.9978 - val_loss: 0.0709 - val_accuracy: 0.9893

 

Save trained model to./models/float_model.h5.

 

Evaluate model on test dataset..

157/157 [==============================] - 1s 3ms/step - loss: 0.0426 - accuracy: 0.9911

loss: 0.043

acc: 0.991

 

 

(vitis-ai-tensorflow2) Vitis-AI /workspace/myproj/tf2-mnist-end-to-end > python quantize.py

Load float model..

 

Load Mnist dataset..

 

Run quantization..

[INFO] Start CrossLayerEqualization...

10/10 [==============================] - 0s 32ms/step

[INFO] CrossLayerEqualization Done.

[INFO] Start Quantize Calibration...

157/157 [==============================] - 12s 77ms/step

[INFO] Quantize Calibration Done.

[INFO] Start Generating Quantized Model...

[Warning] Skip quantize pos adjustment for layer quant_dense, its quantize pos is [i=None, w=8.0, b=9.0, o=3.0]

[INFO] Generating Quantized Model Done.

 

Saved quantized model as ./models/quantized_model.h5

 

 

(vitis-ai-tensorflow2) Vitis-AI /workspace/myproj/tf2-mnist-end-to-end > python eval_quantized.py

 

Load quantized model..

WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually.

 

Load Mnist dataset..

 

Compile model..

 

Evaluate model on test Dataset

157/157 [==============================] - 4s 22ms/step - loss: 0.0417 - accuracy: 0.9913

loss: 0.042

acc: 0.991

 

 

(vitis-ai-tensorflow2) Vitis-AI /workspace/myproj/tf2-mnist-end-to-end > python finetune.py

 

Load float model..

 

Load Mnist dataset..

 

Create quantize training model..

[INFO] Start CrossLayerEqualization...

10/10 [==============================] - 0s 33ms/step

[INFO] CrossLayerEqualization Done.

 

Compiling model..

 

Fit on Dataset..

Epoch 1/10

782/782 [==============================] - 48s 61ms/step - loss: 0.0077 - accuracy: 0.9978 - val_loss: 0.0738 - val_accuracy: 0.9882

Epoch 2/10

782/782 [==============================] - 39s 50ms/step - loss: 0.0062 - accuracy: 0.9980 - val_loss: 0.0845 - val_accuracy: 0.9888

Epoch 3/10

782/782 [==============================] - 40s 51ms/step - loss: 0.0058 - accuracy: 0.9983 - val_loss: 0.0810 - val_accuracy: 0.9885

Epoch 4/10

782/782 [==============================] - 40s 51ms/step - loss: 0.0061 - accuracy: 0.9982 - val_loss: 0.0744 - val_accuracy: 0.9902

Epoch 5/10

782/782 [==============================] - 40s 51ms/step - loss: 0.0048 - accuracy: 0.9984 - val_loss: 0.0834 - val_accuracy: 0.9911

Epoch 6/10

782/782 [==============================] - 39s 50ms/step - loss: 0.0047 - accuracy: 0.9986 - val_loss: 0.0807 - val_accuracy: 0.9893

Epoch 7/10

782/782 [==============================] - 39s 50ms/step - loss: 0.0039 - accuracy: 0.9987 - val_loss: 0.0894 - val_accuracy: 0.9903

Epoch 8/10

782/782 [==============================] - 39s 50ms/step - loss: 0.0034 - accuracy: 0.9989 - val_loss: 0.0863 - val_accuracy: 0.9904

Epoch 9/10

782/782 [==============================] - 39s 49ms/step - loss: 0.0042 - accuracy: 0.9989 - val_loss: 0.1043 - val_accuracy: 0.9893

Epoch 10/10

782/782 [==============================] - 39s 50ms/step - loss: 0.0044 - accuracy: 0.9986 - val_loss: 0.0994 - val_accuracy: 0.9908

 

Saved finetuned model as ./models/finetuned_model.h5

 

Evaluate model on test Dataset..

157/157 [==============================] - 1s 7ms/step - loss: 0.0675 - accuracy: 0.9920

loss: 0.068

acc: 0.992

 

 

(vitis-ai-tensorflow2) Vitis-AI /workspace/myproj/tf2-mnist-end-to-end > bash -x compile_zcu102.sh

+ ARCH=/opt/vitis_ai/compiler/arch/DPUCZDX8G/ZCU102/arch.json

+ OUTDIR=./compiled_model

+ NET_NAME=customcnn

+ MODEL=./models/finetuned_model.h5

+ echo -----------------------------------------

-----------------------------------------

+ echo 'COMPILING MODEL FOR ZCU102..'

COMPILING MODEL FOR ZCU102..

+ echo -----------------------------------------

-----------------------------------------

+ compile

+ tee compile.log

+ vai_c_tensorflow2 --model ./models/finetuned_model.h5 --arch /opt/vitis_ai/compiler/arch/DPUCZDX8G/ZCU102/arch.json --output_dir ./compiled_model --net_name customcnn

/opt/vitis_ai/conda/envs/vitis-ai-tensorflow2/lib/python3.7/site-packages/xnnc/translator/tensorflow_translator.py:1843: H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.

  value = param.get(group).get(ds).value

[INFO] parse raw model     :100%|██████████| 10/10 [00:00<00:00, 16871.70it/s]              

[INFO] infer shape (NHWC)  :100%|██████████| 26/26 [00:00<00:00, 2956.30it/s]               

[INFO] generate xmodel     :100%|██████████| 26/26 [00:00<00:00, 5561.60it/s]               

[INFO] Namespace(inputs_shape=None, layout='NHWC', model_files=['./models/finetuned_model.h5'], model_type='tensorflow2', out_filename='./compiled_model/customcnn_org.xmodel', proto=None)

[INFO] tensorflow2 model: models/finetuned_model.h5

[OPT] No optimization method available for xir-level optimization.

[INFO] generate xmodel: /workspace/myproj/tf2-mnist-end-to-end/compiled_model/customcnn_org.xmodel

[UNILOG][INFO] The compiler log will be dumped at "/tmp/vitis-ai-user/log/xcompiler-20210325-093926-3120"

[UNILOG][INFO] Target architecture: DPUCZDX8G_ISA0_B4096_MAX_BG2

[UNILOG][INFO] Compile mode: dpu

[UNILOG][INFO] Debug mode: function

[UNILOG][INFO] Target architecture: DPUCZDX8G_ISA0_B4096_MAX_BG2

[UNILOG][INFO] Graph name: mnist_customcnn_model, with op num: 42

[UNILOG][INFO] Begin to compile...

[UNILOG][INFO] Total device subgraph number 3, DPU subgraph number 1

[UNILOG][INFO] Compile done.

[UNILOG][INFO] The meta json is saved to "/workspace/myproj/tf2-mnist-end-to-end/./compiled_model/meta.json"

[UNILOG][INFO] The compiled xmodel is saved to "/workspace/myproj/tf2-mnist-end-to-end/./compiled_model/customcnn.xmodel"

[UNILOG][INFO] The compiled xmodel's md5sum is 764142e83d074ea9470b9eb9d0757f68, and been saved to "/workspace/myproj/tf2-mnist-end-to-end/./compiled_model/md5sum.txt"

**************************************************

* VITIS_AI Compilation - Xilinx Inc.

**************************************************

+ echo -----------------------------------------

-----------------------------------------

+ echo 'MODEL COMPILED'

MODEL COMPILED

+ echo -----------------------------------------

-----------------------------------------

 

 

(vitis-ai-tensorflow2) Vitis-AI /workspace/myproj/tf2-mnist-end-to-end > bash -x make_target_zcu102.sh

+ echo -----------------------------------------

-----------------------------------------

+ echo 'MAKE TARGET ZCU102 STARTED..'

MAKE TARGET ZCU102 STARTED..

+ echo -----------------------------------------

-----------------------------------------

+ TARGET_ZCU102=./target_zcu102

+ COMPILE_ZCU102=./compiled_model

+ APP=./application

+ NET_NAME=customcnn

+ rm -rf ./target_zcu102

+ mkdir -p ./target_zcu102/model_dir

+ cp ./application/app_mt.py ./target_zcu102

+ echo '  Copied application to TARGET_ZCU102 folder'

  Copied application to TARGET_ZCU102 folder

+ cp ./compiled_model/customcnn.xmodel ./target_zcu102/model_dir/.

+ echo '  Copied xmodel file(s) to TARGET_ZCU102 folder'

  Copied xmodel file(s) to TARGET_ZCU102 folder

+ mkdir -p ./target_zcu102/images

+ python generate_images.py --dataset=mnist --image_dir=./target_zcu102/images --image_format=jpg --max_images=10000

2021-03-25 09:42:34.445257: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1

Command line options:

 --dataset      :  mnist

 --subset       :  test

 --image_dir    :  ./target_zcu102/images

 --image_list   : 

 --label_list   : 

 --image_format :  jpg

 --max_images   :  10000

+ echo '  Copied images to TARGET_ZCU102 folder'

  Copied images to TARGET_ZCU102 folder

+ echo -----------------------------------------

-----------------------------------------

+ echo 'MAKE TARGET ZCU102 COMPLETED'

MAKE TARGET ZCU102 COMPLETED

+ echo -----------------------------------------

-----------------------------------------

 

ZynqMP评估板上运行

请参阅https://github.com/Xilinx/Vitis-AI/blob/master/setup/mpsoc/VART/README.md#step2-setup-the-target进行电路板设置。之后,将target_zcu10x目录中的所有文件复制到SD卡。从SD卡启动ZCU10x(下面实例运行的是zcu104),运行app_mt.py。结果显示,实际运行模型的精度为99.13%.

 

root@xilinx-zcu104-2020_2:~/target_zcu104# python3 app_mt.py --threads 4

Command line options:

 --image_dir :  images

 --threads   :  4

 --model     :  model_dir/customcnn.xmodel

Pre-processing 10000 images...

Starting 4 threads...

Throughput=5622.61 fps, total frames = 10000, time=1.7785 seconds

Correct:9913, Wrong:87, Accuracy:0.9913