UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Observer anaub408
Observer
1,029 Views
Registered: ‎09-13-2018

Compiling a Non-Convolutional Network

Hello,

is there a way to quantize and compile a tensorflow network without any conv layer inside? Or why is it necessary to have at least one conv layer in the network?
We are trying to get a non-convolutional network running on the dpu. The small network contains the following layers: flatten, dense, activation (relu / softmax),
But while compiling we get the fatal error: [DNNC][Fatal] Check failed for condition [infer_shape_handler != nullptr] in [/tmp/DNNC_V010_Package/dnnc/dnnc_impl/core/layer.cc:162] : Infer shape handler for [Conv] is missing.

It would be really nice to compile networks without any conv layer, because we are facing some problems, were our input data aren't any images.

We are using the dnndk version xlnx_dnndk_v3.0_190430, a tensorflow network, the tensorflow cpu version, ubuntu16.04 and our destination platform is a zcu104 board.

Thanks in advance.

19 Replies
982 Views
Registered: ‎05-17-2019

Re: Compiling a Non-Convolutional Network

I would also like to know whether it is possible to compile a DNN of Fully Connected Layers. It would be very useful.

0 Kudos
Visitor malie431
Visitor
953 Views
Registered: ‎05-20-2019

Re: Compiling a Non-Convolutional Network

I have the same problem. It would be very helpful to compile and run fully connected DNN that were trained in Tensorflow.
Thank you!

0 Kudos
844 Views
Registered: ‎05-17-2019

Re: Compiling a Non-Convolutional Network

I have figured a way to implement Fully Connected Networks using the DNNDK framework.

1) Assume that you have a FC Network with N layers, the size of the 1st Layer is D and your Network Input size is K.

3) Reshape your Network inputs so that they are 2D (kxk). For example if you have 100 inputs reshape them to be 10x10. Apply some form of zero padding if the input size cannot be factored.

2) Replace the 1st FC Layer with a 2D Convolutional Layer with parameters: D filters of size kxk, that is filters that have the same size as your input and are as many as your layer size. This makes the Conv layer to have equivalent operation with a fully connected layer, with the only exception that it operated in 2D. Note: Train your network with the 1st FC layer, not with the Conv layer. I had accuracy loss during training with the Conv layer which may be due to different gradient computation and weight updates. After the training just reshape the weights of your 1st FC layer from KxD to kxkxD and set them to the Conv layer.

3) Apply a Flatten operation to the outputs of the Conv layer and feed them to the rest of your FC Network.

4) If your Network has operations that are not supported by the DNNDK remove them after the training and try to calculate them in CPU.

I am attaching a graph of the first layers of my network (thanks to netron tool). It originally had 81 inputs, which were reshaped to 9x9 and the size of all of its hidden layers is 400. My network was trained with Keras and the model was converted to Tensorflow frozen graph, which was used for the Decent tool.

I hope you find this guide useful and it doesn't violate any of the DNNDK terms.

network.png
Visitor malie431
Visitor
816 Views
Registered: ‎05-20-2019

Re: Compiling a Non-Convolutional Network

Hello mparmpas122321,

great that you found a solution for this problem! I really appreciate that you posted this step-by-step guide.

Is it possible that you share the code of your keras model creation before and after this work around? This would make it even clearer for Keras users like me.

Thanks a lot!

0 Kudos
Observer legomaniak
Observer
484 Views
Registered: ‎03-29-2019

Re: Compiling a Non-Convolutional Network

Hi,

thx mparmpas122321 for solution.

I test it and it works, but it seems to be limited up to 256 input size. I have input size almost 5000. :-( Nevermind, here is my code:

def FCreplace(modelOld):    
    layers = [l for l in modelOld.layers]
    #original input shape train_images Nx1x1x100 but you need to change it to Nx10x10x1
    x = tf.keras.Input(train_images.shape[1:], name="input")
    k=int(np.sqrt(train_images.shape[1] * train_images.shape[2] * train_images.shape[3]))

    net=tf.keras.layers.Conv2D(layers[2].units,kernel_size=(k,k),activation="relu", name='conv2D')(x) 
    net=tf.keras.layers.Flatten(name='flat')(net)  
    
    for i in range(3, len(layers)):
        net = layers[i](net)    
    
    model = tf.keras.Model(inputs=x, outputs=net, name='myNetwork')
    model.layers[1].set_weights([modelOld.layers[2].get_weights()[0].reshape((k,k,1,layers[2].units)), modelOld.layers[2].get_weights()[1]])
    model.summary()
    return model

I had problems with using reshape layer, so you need to rashape input data. Be aware of layers name. 

Good luck to all. 

0 Kudos
Contributor
Contributor
464 Views
Registered: ‎08-02-2019

Re: Compiling a Non-Convolutional Network

Hi legomaniak,

I just want to know what are the layers in the old model you are passing to the function.

Thank you.

0 Kudos
Observer legomaniak
Observer
456 Views
Registered: ‎03-29-2019

Re: Compiling a Non-Convolutional Network

Hi,

Old one is InputLayer(Nx1x1x100)->Flat->Dense0->Dense1-> ...->DenseM 

And new one is InputLayer(Nx10x10x1)->Conv->Flat->Dense1->...->DenseM

After third layers it just copy of your network.

Is it clear?

0 Kudos
Contributor
Contributor
448 Views
Registered: ‎08-02-2019

Re: Compiling a Non-Convolutional Network

Thank you for your reply!

That means if I have input dimension of (N x 50 x 512 x 1) I should reshape it to (N x 160 x 160 x 1) and pass it to Conv2D right!

0 Kudos
439 Views
Registered: ‎05-17-2019

Re: Compiling a Non-Convolutional Network

I am happy that it worked for you! Considering reshape, I had similar issue when I used it as the first layer of my network. Apparently, the first layer of the network must be Convolutional for the tool to work.

I haven't tested it for network input size greater than 81, but I assume that if you reshape your inputs and weights properly, exploiting the 3rd dimension also, you can have up to 256*(16*16)=65536 inputs (?), if we take into account the DPU IP product sepcifications. Correct me if I am wrong.

Screenshot_1.png
0 Kudos
Observer legomaniak
Observer
430 Views
Registered: ‎03-29-2019

Re: Compiling a Non-Convolutional Network

It forks for me with 1x1x100 -> 10x10x1 where kernel size is 10.

I also tested it with 30x30x1 and ended with error where (kernel_size - strides)<=(3*channel_parallel*strides).

where as you mension supported parameters, maximal usable input should be 16x16x1 -> 1x1x256

Is it possible do transformation with 16x16xN size?

thx Lego

0 Kudos
420 Views
Registered: ‎05-17-2019

Re: Compiling a Non-Convolutional Network

Yes, I think it is possible to do the same procedure and reshape input and weights to 16x16xN, where N is the channel dimension.

0 Kudos
Contributor
Contributor
395 Views
Registered: ‎08-02-2019

Re: Compiling a Non-Convolutional Network

Hi,

I have a model as follows:

Layer (type)                         Output Shape             Param #
=================================================================
reshape_1 (Reshape)         (None, 50, 512)               0
_________________________________________________________________
flatten_1 (Flatten)               (None, 25600)                0
_________________________________________________________________
dense_1 (Dense)                   (None, 512)            13107712
_________________________________________________________________
dense_2 (Dense)                   (None, 1024)             525312
_________________________________________________________________
dense_3 (Dense)                    (None, 512)              524800
_________________________________________________________________
dense_4 (Dense)                      (None, 2)                 1026
=================================================================

Is it at all possible to convert the top layer to conv2D and be able to run it in DNNC. If so what should be the input shape? The input shape for the above model is : (N x 50 x 512 x 1)

0 Kudos
381 Views
Registered: ‎05-17-2019

Re: Compiling a Non-Convolutional Network

You should try and replace your 1st dense layer with a conv2D with 512 filters of 16x16x100. You must feed your network with reshaped inputs from (N x 50 x 512 x 1) to (N x 16 x 16 x100). The filter paramaters must be reshaped versions of the 1st dense layer. Of course, remove reshape and flatten layers as dnnc will give error.
Observer legomaniak
Observer
381 Views
Registered: ‎03-29-2019

Re: Compiling a Non-Convolutional Network

Cool it seems working in that way :D thx

0 Kudos
Observer legomaniak
Observer
375 Views
Registered: ‎03-29-2019

Re: Compiling a Non-Convolutional Network

Actualized version of code here:

def FCreplace(modelOld):    
    layers = [l for l in modelOld.layers]
    s=train_images.shape # shape Nx7x7x100
    x = tf.keras.Input(s[1:], name="input")
    net=tf.keras.layers.Conv2D(layers[2].units,kernel_size=(s[1],s[2]), activation="relu", name='conv2D_1')(x) 
    net=tf.keras.layers.Flatten(name='flat')(net)  
    
    for i in range(3, len(layers)):
        net = layers[i](net)    
    
    model = tf.keras.Model(inputs=x, outputs=net, name='myNetwork')
    model.layers[1].set_weights([modelOld.layers[2].get_weights()[0].reshape((s[1],s[2],s[3],layers[2].units)), modelOld.layers[2].get_weights()[1]])
    model.summary()
    return model

For example with these models:

Old model:

Screenshot_1.png

New model:

Screenshot_2.png

0 Kudos
Observer legomaniak
Observer
373 Views
Registered: ‎03-29-2019

Re: Compiling a Non-Convolutional Network

I got error only with use reshape layers, flatten layers are ok.

Thx for help

0 Kudos
Contributor
Contributor
360 Views
Registered: ‎08-02-2019

Re: Compiling a Non-Convolutional Network

Ok so the model would be like:

model = Sequential()
model.add(Reshape((16,16,100), input_shape=(50, 512, 1)))
model.add(Conv2D(100,(16,16), padding="same", activation="relu"))
model.add(GlobalAveragePooling2D())
model.add(Dense(512))
model.add(Dense(512))
model.add(Dense(512))
model.add(Dense(self.nb_classes, activation='softmax'))

But how to remove Reshape layer because it is necessary for reshaping input?

0 Kudos
268 Views
Registered: ‎05-17-2019

Re: Compiling a Non-Convolutional Network

You can just reshape the inputs before you feed them to the network. For example,

....

import numpy as np

x = np.reshape(x, (...))

y = model.predict(x)

Contributor
Contributor
241 Views
Registered: ‎08-02-2019

Re: Compiling a Non-Convolutional Network

Oh yes! Ofcourse! Thank you.

0 Kudos