cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
2,311 Views
Registered: ‎03-23-2019

DNNC v2.08 potential shape calculation bug for deconvolution layers

Jump to solution

So after modifying my prototxt heavily to fit the network within the operations allowed by DNNDK, I was finally able to quantize my custom trained Yolov3-tiny network with DECENT. However, the DNNC step fails, saying that the shape of the blobs do not match for concatenation.

 

$ ./decent.sh
[...]
I0323 03:08:33.302323 17427 net.cpp:330] Network initialization done.
I0323 03:08:33.341049 17427 decent.cpp:333] Start Deploy
I0323 03:08:33.449362 17427 decent.cpp:341] Deploy Done!
--------------------------------------------------
Output Deploy Weights: "[...]/xilinx_dnndk_v2.08/host_x86/models/yolov3-tiny/decent_output/deploy.caffemodel"
Output Deploy Model: "[...]/xilinx_dnndk_v2.08/host_x86/models/yolov3-tiny/decent_output/deploy.prototxt"

$ ./dnnc.sh Compiling network: yolov3_tiny_c1 [DNNC][Error] Make sure concat's inputs all have same width and height shape, current concat layer is [layer21_route]. [DNNC][Error] Infer shape for current layer [layer21_route] failed.

 

But surely they do match, since I was able to run inference on my host computer with Caffe and it passes the DECENT step. The offending layer is a concatenation of two values at the channel layer, one of which is the output of a deconvolution operation. The deconvolution operation was used to implement the upsampling layer for Yolov3-tiny.

My guess is that DNNC is miscomputing the output shape of the deconvolution, making it think that there's a shape mismatch. Is there any way I can test if this is happening and work around it? Alternatively, is there a way to split the network at the upsampling stage so that I can do that step on the CPU instead? If I just split the prototxt into two files, I have no way to calibrate the lower half when running DECENT since the input data won't be images.

I would attach my model folder if I could, but I think the forum isn't letting me post when I do that.

0 Kudos
Reply
1 Solution

Accepted Solutions
2,268 Views
Registered: ‎03-23-2019

I found the reason. It was because I had large Slice and Concat layers, which somehow affected the ability of DNNC to compute the proper blob shapes for downstream layers. When I replace the Slice and Concat layers with convolutions (truly quite inefficient, but what can a man do?), the DNNC phase passes.

View solution in original post

0 Kudos
Reply
8 Replies
2,269 Views
Registered: ‎03-23-2019

I found the reason. It was because I had large Slice and Concat layers, which somehow affected the ability of DNNC to compute the proper blob shapes for downstream layers. When I replace the Slice and Concat layers with convolutions (truly quite inefficient, but what can a man do?), the DNNC phase passes.

View solution in original post

0 Kudos
Reply
qhall
Explorer
Explorer
2,206 Views
Registered: ‎10-24-2008

dalud@andrew.cmu.eduThis is interesting work.  Thanks for sharing this.  Did you employ the Xilinx DarkNet to Caffe tool for this effort?

--Quenton

0 Kudos
Reply
ahe
Xilinx Employee
Xilinx Employee
2,197 Views
Registered: ‎03-02-2017

dalud@andrew.cmu.edu  已写:

I found the reason. It was because I had large Slice and Concat layers, which somehow affected the ability of DNNC to compute the proper blob shapes for downstream layers. When I replace the Slice and Concat layers with convolutions (truly quite inefficient, but what can a man do?), the DNNC phase passes.


So you change the model structure that may cause the accuracy changed? Have you got the test accuracy compared with the origian yolo mode?

Thanks,

Alex

0 Kudos
Reply
2,170 Views
Registered: ‎03-23-2019

I've used this repo to convert the Yolov3-tiny darknet model to caffe, but I had to make some significant changes on top of it.

https://github.com/ChenYingpeng/caffe-yolov3

I first tried to implement upsample with reshapes and tiling, but DNNC didn't support those operations. Then I used deconvolution. I originally implemented the unevenly padded max-pooling layer with an evenly padded max-pooling layer + slice into 14 columns and concat 13 of them, but I had the bug mentioned earlier so I replaced the slice and concat layers with a 2x2 convolution.

 

I haven't tested the loss, but I adapted the face detection demo to use Yolov3 to detect pedestrians and it behaves pretty well there (though the discretization is causing the bounding box sizes to visibly flicker). I didn't need to re-train, since I was able to use the trained weights even after the modifications to the structure as the final computation outcome remains the same.

0 Kudos
Reply
aluo
Xilinx Employee
Xilinx Employee
2,160 Views
Registered: ‎02-18-2013

dalud@andrew.cmu.eduit is cool. Are you able to share the modified model? 

0 Kudos
Reply
jsi_wmz
Contributor
Contributor
2,140 Views
Registered: ‎08-09-2018

dalud@andrew.cmu.edu  In yolov3-tiny model, I replace upsample layer with deconvolution. It passed the decent script. But an error occurred when it came to dnnc, like this

 

./dnnc_yolov3-tiny.sh 
Compiling network: yolov3tiny
[DNNC][Fatal] Check failed for condition [current->in_degree() > 1] in [/tmp/DNNC_V010_Package/dnnc/dnnc_impl/core/layergraph.cc:111] :
*** Check failure stack trace: ***

Does anyone know the meaning of this error?

 

My deconv layer wrote like this

 

layer {
  name: "layer20-deconv"
  type: "Deconvolution"
  bottom: "layer19-conv"
  top: "layer20-deconv"
  phase: TRAIN
  convolution_param {
    num_output: 128
    bias_term: false
    pad: 0
    kernel_size: 2
    stride: 2
  }
}

 

 

0 Kudos
Reply
2,122 Views
Registered: ‎03-23-2019

I have the input prototxt included. This is for a modified version of Yolov3-tiny with only one output class (which changes the number of filters at some layers from 255 to 18).

Usually I don't find the error messages useful. Instead, it helps to remove layers one at a time from the bottom until the error disappears; the last one removed would be the one that DNNC doesn't like.

0 Kudos
Reply
jsi_wmz
Contributor
Contributor
2,091 Views
Registered: ‎08-09-2018

dalud@andrew.cmu.edu Thanks a lot for your help. I just removed the layer one by one and eventually found the reason. In original yolov3 tiny, there is a route(concat) layer which has only one input.  The concat layer has one input is allowed in Caffe but not DNNC. After i removed that route layer, it worked.

0 Kudos
Reply