cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Contributor
Contributor
939 Views
Registered: ‎07-05-2018

How to know the accuracy of quantised model using Deephi tool

Jump to solution

Hi,

I am using the decent tool for quantisation of Resnet model to 8 bit int.

The ouput is generated as deploy.prototxt and deploy.caffemodel.

I want to know the accuracy of the quantised model like the screenshot attached.
To see the accuracy, the user guide says that in decent tool, I need to set the auto_test option to TRUE.
It also says that I need to provide the test dataset to know the accuracy. Can you tell me where to set the path for the test dataset or how to provide test dataset to model so that it can provide me accuracy for quantised model?
 
Regards,
Shikha Goel
(Ph.D. , IIT Delhi)
accuracy.png
0 Kudos
1 Solution

Accepted Solutions
Highlighted
Explorer
Explorer
873 Views
Registered: ‎10-24-2008

@anz162112This tutorial illustrates that the calibration images directory is actually a parameter which is set in the float.prototxt file. 

Ultra96_ML_Embedded_Workshop

This example also uses the auto_test function.  You should be able to refer to the ResNet example as a guide if you have either a ZCU102, 104 or Ultra96 as a starting point, and have loaded the latest stretch image and board files from the Developer Hub:

https://www.xilinx.com/products/design-tools/ai-inference/ai-developer-hub.html#edge

--Quenton

 

View solution in original post

0 Kudos
5 Replies
Highlighted
Explorer
Explorer
874 Views
Registered: ‎10-24-2008

@anz162112This tutorial illustrates that the calibration images directory is actually a parameter which is set in the float.prototxt file. 

Ultra96_ML_Embedded_Workshop

This example also uses the auto_test function.  You should be able to refer to the ResNet example as a guide if you have either a ZCU102, 104 or Ultra96 as a starting point, and have loaded the latest stretch image and board files from the Developer Hub:

https://www.xilinx.com/products/design-tools/ai-inference/ai-developer-hub.html#edge

--Quenton

 

View solution in original post

0 Kudos
Highlighted
Contributor
Contributor
860 Views
Registered: ‎07-05-2018

I could get this working using the link you mentioned.

 

Thanks

Shikha Goel

Regards,
Shikha Goel
(Ph.D. , IIT Delhi)
0 Kudos
Highlighted
Visitor
Visitor
794 Views
Registered: ‎06-20-2018

Hello,

I followed step specified in the below link to check the accuracy for decent [quantization]

https://github.com/jimheaton/Ultra96_ML_Embedded_Workshop

Without auto_test, we are able to generate deploy.prototxt and deploy.caffemodel.

With auto_test, we are facing error, Please find the attachment. Kindly provide the solution 

Regards

Manoj

error1.JPG
0 Kudos
Highlighted
Explorer
Explorer
776 Views
Registered: ‎10-24-2008

@bmanoj19You will need to add a data layer for TEST as shown:

 

name: "ResNet-50"
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
mirror: true
crop_size: 224
mean_value: 104
mean_value: 107
mean_value: 123
}
image_data_param {
source: "./data/imagenet_val/calibration.txt"
root_folder: "./data/imagenet_val/calibration_images/"
batch_size: 10
shuffle: false
}
}
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
include {
phase: TEST
}
transform_param {
mirror: false
crop_size: 224
mean_value: 104
mean_value: 107
mean_value: 123
}
image_data_param {
source: "./data/imagenet_val/calibration.txt"
root_folder: "./data/imagenet_val/calibration_images/"
batch_size: 20
shuffle: false
}
}
layer {
name: "conv1"
type: "Convolution"

.............

--Quenton

0 Kudos
Highlighted
Visitor
Visitor
753 Views
Registered: ‎06-20-2018

I could able to run and get the accuracy for resent50. Thank you for Inputs

Regards

Manoj B

0 Kudos