Can not assign global port to HBM bank other than [0] when using hls:stream interface in SW_EMU mode

[doonny]

I have found a incompatible features of using hls:stream interfaces with global memory interfaces in sw emu mode.

For instance, here's two kernels:  DataLoad and DataStore. DataLoad is designed to fetch data from independent HBM banks and pass the data stream to DataStore, which then store the data back to a different HBM banks.  The codes are as following:

--------------------------------------

#include "ap_axi_sdata.h"

#include "ap_int.h"

#include "hls_stream.h"

#define VEC_SIZE    16

typedef ap_uint<VEC_SIZE*32> float_vec;

typedef ap_axiu<VEC_SIZE*32,0,0,0> k2k_data;

extern "C" {

void DataLoad(

                const float_vec *A_in,  //HBM[0]

                const float_vec *C_in,  //HBM[2]

                const unsigned int data_num,

                hls::stream<k2k_data> &stream_out_0,

                hls::stream<k2k_data> &stream_out_1

                )

{

#pragma HLS INTERFACE m_axi port = A_in  offset = slave bundle = gmem0 // group-0

#pragma HLS INTERFACE m_axi port = C_in  offset = slave bundle = gmem1 // group-1

    float_vec tmp2;

    k2k_data tmp;

    for(unsigned int i=0; i<data_num; i++){

        tmp2 = A_in[i];

        tmp.data = tmp2;

        stream_out_0.write(tmp);

        tmp2 = C_in[i];

        tmp.data = tmp2;

        stream_out_1.write(tmp);

    }

}

}

extern "C" {

void DataStore(

                float_vec *B_out,       //HBM[1]

                float_vec *D_out,       //HBM[3]

                const unsigned int data_num,

                hls::stream<k2k_data> &stream_in_0,

                hls::stream<k2k_data> &stream_in_1

                )

{

#pragma HLS INTERFACE m_axi port = B_out offset = slave bundle = gmem2 // group-0

#pragma HLS INTERFACE m_axi port = D_out offset = slave bundle = gmem3 // group-1

    k2k_data tmp;

    for(unsigned int i=0; i<data_num; i++){

        tmp = stream_in_0.read();

        B_out[i] = tmp.data;

        tmp = stream_in_1.read();

        D_out[i] = tmp.data;

    }

}

}

-------------------------------------------------------

It was found that when performing sw emulation, all buffers could only be allocated to HBM[0], otherwise, the following ERROR will show up:

[XRT] WARNING: Argument '1' of kernel 'DataLoad' is allocated in memory bank 'HBM[0]'; compute unit 'DataLoad_1' cannot be used with this argument and is ignored.
[XRT] ERROR: kernel 'DataLoad' has no compute units to support required argument connectivity.
ERROR: clSetKernelArg() for kernel "DataLoad", argument index 1.
host/main.cpp:184 Error calling err = krnl_load.setArg(narg++, buffer_c), error code is: -4

Attached is the Vitis project, in which I have forced all buffers to be allocated in HBM[0]. And if not, the above erros will show up.

[nutang]

Hi @doonny 

The description is incomplete.

Kindly provide more details about the platform, steps and references

[doonny]

Hi @nutang

I am using the U50 card on a Ubuntu 18.04.4 LTS machine with Vitis 2020.1 SDK.

I wrote a simple program to test the use of streaming interfaces between two kernels like this:

When performing sw emulation, the runtime give error showing that DataLoad only has access to HBM[0] and host program could not locate buffers in HBM[2].  But the same program works fine on FPGA board with no error.  It seems that the runtime failed to connect DataLoad kernel with HBM[2] when performing emulation.

And this error only show when I added the hls::streaming interfaces between two kernels.  If I use only One Kernel to fetch and store data (like the hbm_simple exampel did) there will not be any issues when performing emulation.

So my guess was that the hls::stream interface has issue with the HBM multi-bank configuration in emulation runtime.