cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
nico.teringl
Observer
Observer
675 Views
Registered: ‎02-12-2020

VAI_C_TENSORFLOW ERROR: "Invalid Reshape size"

Hello,

our custom (fully connected) Tensorflow model successfully passed the Quantization phase, but the Compiler gives us the following error:

[VAI_C][Error] Invalid Reshape size: input size is [100352], reshape size is [25088], current op is [time_distributed_2_Reshape].
[VAI_C][Error] Infer shape for current layer [time_distributed_2_Reshape] failed.

The input shape of the input node is: (1,1,4,25088).

We assume that the input size is too big for the DPU! (We use a Xilinx MPSoC Ultrascale+ with DPUv2, B4096)

We can provide the PB files, if necessary!

For now, we added a picture of the structure of the original model, and the output model from the Quantizer!

Please, can you help us finding a solution, we are already trying to bring our network on the DPU for a few months now!

 

This is the original model:

model_cls_separate_without_dropout.pb.png

This is the output of the Quantizer (Deploy Model

deploy_model_class_sep.pb.png

0 Kudos
6 Replies
jasonwu
Moderator
Moderator
582 Views
Registered: ‎03-27-2013

Hi @nico.teringl ,

 

It is strange for me that we can't find reshape size limation here.

Let me try to find someone for help.

Best Regards,
Jason
-----------------------------------------------------------------------------------------------
Please mark the Answer as "Accept as solution" if the information provided is helpful.

Give Kudos to a post which you think is helpful and reply oriented.
-----------------------------------------------------------------------------------------------
nico.teringl
Observer
Observer
560 Views
Registered: ‎02-12-2020

Hello,

thank you very much. We appreciate every kind of support as we're already working on this network for a few months now, and we have to finally find a solution soon!

Best regards,

Nico

0 Kudos
nico.teringl
Observer
Observer
540 Views
Registered: ‎02-12-2020

Hi @jasonwu ,

we further analyzed the processed models, and found out that, apparently, the Quantizer Tool does not correctly process the first "Squeeze" operation, or the first "Reshape" operation. We are not sure, where the exact problem lays, but the following two images show the difference between the output graphs of the Quantizer Tool ("Deploy Model" and "Qunatize Eval Model").

In the original model, the parameter "shape" of the "Reshape" operation is [-1, 4608], and in the "Quantize Eval Model", this parameter is equal. But if we open the "Deploy Model" PB file with the tool netron, the "shape" parameter apparently changed to [+1, 4608]!

The orig graph:

orig.png

 

The "Quantize Eval Model":

eval.png

 

The "Deploy Model" (note that in this model, the "shape" parameter is different: [1, 4608], instead of [-1, 4608]!

deploy.png

 

May this cause problems, when processed by the VAI Compiler?

 

Thank you,

Best regards,

Nico

0 Kudos
jasonwu
Moderator
Moderator
509 Views
Registered: ‎03-27-2013

Hi @nico.teringl ,

 

I have send the error message to our dev contact and it seems that we still can't locate the issue.

Is it possible to send your pb file for further investigation?

If so please provide your email address(you can send me a private message).

I would send you a EZMOVE request so that you can send your pb to me safely.

Best Regards,
Jason
-----------------------------------------------------------------------------------------------
Please mark the Answer as "Accept as solution" if the information provided is helpful.

Give Kudos to a post which you think is helpful and reply oriented.
-----------------------------------------------------------------------------------------------
jasonwu
Moderator
Moderator
445 Views
Registered: ‎03-27-2013

Hi @nico.teringl ,

 

Got some feedback from dev team according to the resouce you provided:

This model is a little bit strange at the beginning. The input node has shape of [-1, -1, 4, 25088] and the reshape node has shape [-1, 25088]. The batch_size dimension(-1) and part of the shape dimension (-1, 4, 25088) are mixed during reshaping, making the batch_size information disappeared.

Please confirm that whether the output shape of the reshape nodes is correct and is exactly what the algorithm designer wants. I think the reshape output shape should be [-1, 4*25088].

Best Regards,
Jason
-----------------------------------------------------------------------------------------------
Please mark the Answer as "Accept as solution" if the information provided is helpful.

Give Kudos to a post which you think is helpful and reply oriented.
-----------------------------------------------------------------------------------------------
nico.teringl
Observer
Observer
419 Views
Registered: ‎02-12-2020

Hello Jason,

thank you for your answer!

This graph is the last stage of a faster RCNN structure. The original part had only 3 input dimensions.

Since the quantifier needs 4 dimensions, we added another dimension before the input as a workaround. This fourth dimension gets squeezed out afterwards, in order to keep the original workflow of the graph.

The problematic reshape function seems to come from a keras time_distributed layer.  Are these time_distributed layers from keras supported, or is there a problem with them?

 

Best regards,

Nico

0 Kudos