02-16-2019 01:33 AM
I want to quantise the resnet 32 bit float model to 16 bit int and run it on ZCU102 using Deephi.
Steps I followed are:
1. First I used Decent tool to quantise the model to 16 bit int by changing weight_bit and data_bit to 16 in script. The deploy.prototxt and deploy.caffemodel were generated.
2. Then used DNNC tool to generate elf files for it.
3. Then on ZCU102, installed Deephi. and in samples/model folder, replaced the earlier model files with new model files(the one I generated). I kept the source code(main.cc) same.
4. Run make command. Then run ./resnet.
But the accuracy of 16 bit quantised model is very poor(almost 0) as compared to accuracy of 8 bit quantised model.
I have attached the output of both 8 bit and 16 bit model on same image.
This behaviour is strange. Am I doing something wrong? Do I need to change anything in main.cc for 16 bit inference?
02-16-2019 06:17 AM
Currently, the DPU processor is designed for 8-bit quantizataion model only.
Therefore you are getting zero accuracy for the 16-bit model
02-17-2019 02:46 AM
Okay. Actually in user guide it says that we can quantise to any bit width. So, I wanted to try it for my work.