cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
anz162112
Contributor
Contributor
1,243 Views
Registered: ‎07-05-2018

How to know the accuracy of quantised model using Deephi tool

Jump to solution

Hi,

I am using the decent tool for quantisation of Resnet model to 8 bit int.

The ouput is generated as deploy.prototxt and deploy.caffemodel.

I want to know the accuracy of the quantised model like the screenshot attached.
To see the accuracy, the user guide says that in decent tool, I need to set the auto_test option to TRUE.
It also says that I need to provide the test dataset to know the accuracy. Can you tell me where to set the path for the test dataset or how to provide test dataset to model so that it can provide me accuracy for quantised model?
 
Regards,
Shikha Goel
(Ph.D. , IIT Delhi)
accuracy.png
0 Kudos
1 Solution

Accepted Solutions
qhall
Explorer
Explorer
1,177 Views
Registered: ‎10-24-2008

@anz162112This tutorial illustrates that the calibration images directory is actually a parameter which is set in the float.prototxt file. 

Ultra96_ML_Embedded_Workshop

This example also uses the auto_test function.  You should be able to refer to the ResNet example as a guide if you have either a ZCU102, 104 or Ultra96 as a starting point, and have loaded the latest stretch image and board files from the Developer Hub:

https://www.xilinx.com/products/design-tools/ai-inference/ai-developer-hub.html#edge

--Quenton

 

View solution in original post

0 Kudos
5 Replies
qhall
Explorer
Explorer
1,178 Views
Registered: ‎10-24-2008

@anz162112This tutorial illustrates that the calibration images directory is actually a parameter which is set in the float.prototxt file. 

Ultra96_ML_Embedded_Workshop

This example also uses the auto_test function.  You should be able to refer to the ResNet example as a guide if you have either a ZCU102, 104 or Ultra96 as a starting point, and have loaded the latest stretch image and board files from the Developer Hub:

https://www.xilinx.com/products/design-tools/ai-inference/ai-developer-hub.html#edge

--Quenton

 

View solution in original post

0 Kudos
anz162112
Contributor
Contributor
1,164 Views
Registered: ‎07-05-2018

I could get this working using the link you mentioned.

 

Thanks

Shikha Goel

Regards,
Shikha Goel
(Ph.D. , IIT Delhi)
0 Kudos
bmanoj19
Visitor
Visitor
1,098 Views
Registered: ‎06-20-2018

Hello,

I followed step specified in the below link to check the accuracy for decent [quantization]

https://github.com/jimheaton/Ultra96_ML_Embedded_Workshop

Without auto_test, we are able to generate deploy.prototxt and deploy.caffemodel.

With auto_test, we are facing error, Please find the attachment. Kindly provide the solution 

Regards

Manoj

error1.JPG
0 Kudos
qhall
Explorer
Explorer
1,080 Views
Registered: ‎10-24-2008

@bmanoj19You will need to add a data layer for TEST as shown:

 

name: "ResNet-50"
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
include {
phase: TRAIN
}
transform_param {
mirror: true
crop_size: 224
mean_value: 104
mean_value: 107
mean_value: 123
}
image_data_param {
source: "./data/imagenet_val/calibration.txt"
root_folder: "./data/imagenet_val/calibration_images/"
batch_size: 10
shuffle: false
}
}
layer {
name: "data"
type: "ImageData"
top: "data"
top: "label"
include {
phase: TEST
}
transform_param {
mirror: false
crop_size: 224
mean_value: 104
mean_value: 107
mean_value: 123
}
image_data_param {
source: "./data/imagenet_val/calibration.txt"
root_folder: "./data/imagenet_val/calibration_images/"
batch_size: 20
shuffle: false
}
}
layer {
name: "conv1"
type: "Convolution"

.............

--Quenton

0 Kudos
bmanoj19
Visitor
Visitor
1,057 Views
Registered: ‎06-20-2018

I could able to run and get the accuracy for resent50. Thank you for Inputs

Regards

Manoj B

0 Kudos