UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

Problems in using multiple DDR banks in HLS based sdx project

Accepted Solution Solved
Reply
Highlighted
Explorer
Posts: 110
Registered: ‎06-17-2012
Accepted Solution

Problems in using multiple DDR banks in HLS based sdx project

I am trying to make use of the 4 DDR banks in ku115 in my HLS based sdx project.

I see examples in the github, but it is using c++ based opencl bindings.

When I tried to port it to my design using c based opencl bindings.

The design compiles just fine, but I got the following runtime errors.

 

XCL_EMULATION_MODE=sw_emu ./host
INFO: Importing xclbin/mem_test.sw_emu.xilinx_xil-accel-rd-ku115_4ddr-xpr.xclbin
INFO: Loaded file
INFO: Created Binary
INFO: Built Program
ERROR: bad host_ptr of mem use flags
ERROR: bad host_ptr of mem use flags
ERROR: bad host_ptr of mem use flags
ERROR: bad host_ptr of mem use flags
Segmentation fault (core dumped)

 

And the error information is too simple to debug. The design is simple and it works if I don't specify the DDR bank.

Here is the device memory declaration part in the host.

 

cl_mem_ext_ptr_t in_ext0, in_ext1, in_ext2, in_ext3;
cl_mem_ext_ptr_t out_ext0, out_ext1, out_ext2, out_ext3;

in_ext0.flags = XCL_MEM_DDR_BANK0; 
in_ext1.flags = XCL_MEM_DDR_BANK1; 
in_ext2.flags = XCL_MEM_DDR_BANK2; 
in_ext3.flags = XCL_MEM_DDR_BANK3;

in_ext0.obj = input0; // 0?
in_ext1.obj = input1; // 0?
in_ext2.obj = input2; // 0?
in_ext3.obj = input3; // 0?

in_ext0.param = 0; 
in_ext1.param = 0; 
in_ext2.param = 0; 
in_ext3.param = 0;

cl_mem devMemInput0 = clCreateBuffer(world.context,
CL_MEM_READ_ONLY | CL_MEM_EXT_PTR_XILINX, sizeof(int) * MEM_SIZE, &in_ext0, &err);
cl_mem devMemInput1 = clCreateBuffer(world.context,
CL_MEM_READ_ONLY | CL_MEM_EXT_PTR_XILINX, sizeof(int) * MEM_SIZE, &in_ext1, &err);
cl_mem devMemInput2 = clCreateBuffer(world.context,
CL_MEM_READ_ONLY | CL_MEM_EXT_PTR_XILINX, sizeof(int) * MEM_SIZE, &in_ext2, &err);
cl_mem devMemInput3 = clCreateBuffer(world.context,
CL_MEM_READ_ONLY | CL_MEM_EXT_PTR_XILINX, sizeof(int) * MEM_SIZE, &in_ext3, &err);

 

Here is the makefile CLFPGAS setup.

I am not sure how does this configuration come from, but I guess this can be the problem.

 

mem_test_LDCLFLAGS=

--xp misc:map_connect=add.kernel.mem_test_1.M_AXI_GMEM0.core.OCL_REGION_0.M00_AXI

--xp misc:map_connect=add.kernel.mem_test_1.M_AXI_GMEM1.core.OCL_REGION_0.M01_AXI

--xp misc:map_connect=add.kernel.mem_test_1.M_AXI_GMEM2.core.OCL_REGION_0.M02_AXI

--xp misc:map_connect=add.kernel.mem_test_1.M_AXI_GMEM3.core.OCL_REGION_0.M03_AXI

 

Any suggestions are appreciated.

 

regards,

Cheng Liu


Accepted Solutions
Moderator
Posts: 249
Registered: ‎03-27-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Hi Cheng,

 

Just let you know that I made a simple test code using 16 ports and it woks for me on 2017.2.

Of cause, the "--max_memory_ports" xocc option was removed.

 

Regards,

Sean

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------

View solution in original post


All Replies
Contributor
Posts: 26
Registered: ‎12-23-2015

Re: Problems in using multiple DDR banks in HLS based sdx project

 

Did you not forget to specify a different "bundle"  in your HLS code? 

 

#pragma HLS INTERFACE m_axi port=input  offset=slave bundle=gmem0
#pragma HLS INTERFACE m_axi port=output offset=slave bundle=gmem1

I tried to use multiple DDR banks according to these links (host, kernel) and everything worked fine.

Explorer
Posts: 110
Registered: ‎06-17-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Thanks for the information.

According to the reference, I found that I didn't pass CL_MEM_USE_HOST_PTR while setting

the obj attribute to be the host pointer.

 

By the way, it seems that we can only map the whole memory bundle to one of the memory bank.

In my design, the kernel function has four arrays stored in global memory. And I want to spread each arrays to the

4 memory banks such that the memory bandwidth utilization can be improved. To that end, I divide each array into

four sub arrays and each of the sub array must be specified with an independent memory bundle in the kernel. As a result,

I have 16 memory bundles in the kernel which reaches the memory bundle limit of the design (16 master ports).

So is there a way to allocate the data into the different memory banks without affecting the number of memory bundles?

 

Regards,

Cheng Liu 

Moderator
Posts: 249
Registered: ‎03-27-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Have you considered to combine sub arrays with struct and use DATA_PACK on it?

 

For  example, for 4 port A, B, C, D, 

typedef struct{
A1;
B1;
C1;

D1;
} port1;

 

typedef struct{
A2;
B2;
C2;

D2;
} port2;

 

...

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
Explorer
Posts: 110
Registered: ‎06-17-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Hi, @seanz

 

Thanks for the reply.

DATA PACK makes the design more compact and clean, but it will not solve this problem.

In fact, I can have the four sub arrays sharing the same the global memory port.

However, I can only map this memory port to one memory bank using the command.

--xp misc:map_connect=add.kernel.bfs_1.M_AXI_GMEM1.core.OCL_REGION_0.M01_AXI  

 

While my original plan is to put the sub arrays into different memory banks.

 

Regards,

Cheng Liu

 

 

Moderator
Posts: 249
Registered: ‎03-27-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Hi @liucheng,

 

 

I am not quite understand what you mean. 

Can you post some pseudo-code?

 

Regards,

Sean

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
Explorer
Posts: 110
Registered: ‎06-17-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Hi, @seanz,

Here is a simple example.

My kernel has 4 pipelined sub functions.

 

void kernel(int * A, int* B, int* C, int*D, int* X){

 #paragma HLS DATAFLOW

fun1(A, ...);

fun2(B, ...);

fun3(C, ...);

fun4(D, ...);

}

 

 

Now I want to split A, B, C and D to the 4-bank DDR, so they 

can benefit form multiple bank memory.

Then kernel is changed to the following format:

void kernel(int * A, int* B, int* C, int*D, int* X){

 #paragma HLS DATAFLOW

fun1(A_Bank0, A_Bank1, A_Bank2, A_Bank3, ...);

fun2(B_Bank0, B_Bank1, B_Bank2, B_Bank3, ...);

fun3(C_Bank0, C_Bank1, C_Bank2, C_Bank3, ...);

fun4(D_Bank0, D_Bank1, D_Bank2, D_Bank3, ...);

}

 

If I understand correctly, I need 16 master ports to implement this design.

And there is no port left for any other master ports.

So my question is whether there is a way to save the use of the

master ports in a multiple-bank design.  For example,

is it possible to have A_Bank0, A_Bank1 A_Bank2, A_Bank3 share the same

master ports but still mapped to different memory banks?

 

Regards,

Cheng Liu

Moderator
Posts: 249
Registered: ‎03-27-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Hi @liucheng

 

Do you have "--max_memory_ports" xocc option in the makefile? 

Remove it and then ports can share memory banks.

 

Regards,

Sean

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
Moderator
Posts: 249
Registered: ‎03-27-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Hi Cheng,

 

Just let you know that I made a simple test code using 16 ports and it woks for me on 2017.2.

Of cause, the "--max_memory_ports" xocc option was removed.

 

Regards,

Sean

-------------------------------------------------------------------------
Don’t forget to reply, kudo, and accept as solution.
-------------------------------------------------------------------------
Explorer
Posts: 110
Registered: ‎06-17-2012

Re: Problems in using multiple DDR banks in HLS based sdx project

Hi, @seanz,

 

Thanks for the kind reply and help.

Yes, it works all right now.

 

Regards,

Cheng Liu