UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Visitor jadeanderson
Visitor
5,401 Views
Registered: ‎03-23-2015

PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

I am trying use the Xilinx PCIE DMA Subsytem with my KCU105 board.  The problem I have is that when I use the Card2Host AXI streaming channel, I get more data than I expect.

I haven’t found a way to get data back to the host without generating an AXI TLAST pulse.  So I have a counter that counts the number of 32byte AXI4-S  Data transfers, then it generates a pulse on the Tlast input of the DMA C2H port.   I must be doing this incorrectly b/c when I send the pulse, the DMA should sense that the transfer is finished, and then stop the request for data.  

I have data flowing from the ADC and it is valid every clock cycle, so I need to get that into the HOST and do so without losing samples.  I have plenty of bandwidth on the FPGA to Host.

I have not figured out how to do this cleanly.  I get the following:   

 

I have my transfer size and my internal FPGA counter set exactly the same, so for a 131072 byte transfer I should get 1 tlast pulse.

 

 

 ./dma_from_device -d /dev/xdma0_c2h_0 -f output_data.bin -s 131072 -c 1

 

sscanf() = 1, value = 0x00020000

sscanf() = 1, value = 0x00000001

device = /dev/xdma0_c2h_0, address = 0x00000000, size = 0x00020000, offset = 0x00000000, count = 1

host memory buffer = 0x7f9639ad3000

CLOCK_MONOTONIC reports 0.000112370 seconds (total) for last transfer of 131072 bytes

Done

 

Tail OF THE DMESG LOG on Linux HOST PC

 

[105773.751115] ch_irq = 0x00000002

[105773.751126] ch_irq = 0x00000002

[105773.751136] ch_irq = 0x00000002

[105773.751146] ch_irq = 0x00000002

[105773.751157] ch_irq = 0x00000002

[105773.751167] ch_irq = 0x00000002

[105773.751177] ch_irq = 0x00000002

[105773.751187] ch_irq = 0x00000002

[105773.751197] ch_irq = 0x00000002

[105773.751208] ENTERING char_sgdma_close,total read=370

[105773.751209] ch_irq = 0x00000002

[105773.751218] ch_irq = 0x00000002

 

The ILA in my design shows that the c2h_s_axis_tready line is ready for more data,  I thought it should go low and stay low after the transfer was complete. The main problem is that I am blindly generating the TLAST signal based on a hard coded transfer size and I can only request that size in the host.  There must be a better way to send data samples from FPGA to Host with this C2H streaming interface. If not, then at the least, I need clean way to reset the TLAST counter at the beginning of a transfer.  If I knew the number of bytes requested, then I could load the counter value at the beginning of the transfer and then send the tlast pulse at the exact boundary.

 

Thanks

 

 

Tags (1)
0 Kudos
1 Solution

Accepted Solutions
Xilinx Employee
Xilinx Employee
7,853 Views
Registered: ‎05-07-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

HI @jadeanderson

 

Disabling cyclic buffer  might help your use case.
please replace ther xdma-core.c file in AR65444 driver with the the one attached  and then load the driver and run the tests  once

Thanks
Bharath
--------------------------------------------------​--------------------------------------------
Please mark the Answer as "Accept as solution" if information provided addresses your query/concern.
Give Kudos to a post which you think is helpful.
--------------------------------------------------​-------------------------------------------
15 Replies
Moderator
Moderator
5,379 Views
Registered: ‎02-16-2010

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution
s_axis_c2h_tready_x is an indication that DMA is ready to accept the data. You have to assert s_axis_c2h_tvalid_x HIGH whenever you have valid data. If you do not have valid data to send, you may assert s_axis_c2h_tvalid_x LOW.

When you finish the transfer of data, you can assert s_axis_c2h_tlast_x HIGH in the last cycle of the data and assert s_axis_c2h_tvalid_x to be LOW from the next cycle.
------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------
0 Kudos
Visitor jadeanderson
Visitor
5,340 Views
Registered: ‎03-23-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

I tried doing this, but the problem that I face now is that there are so many channel Interrupts that it locks up the host. So can you point me to a simple AXI streaming generator that I can try ?  I built my own, but I don't know how to control the fact that I always have valid data since my application is ADC > FIFO > PCIE_DMA

I was going to mimic this with a simple verliog model.

drive t_data port with a counter

drive t_valid and t_last based on the rising edge of the t_ready from the host.

See attached file.

 

Also, can I am trying to use the latest Linux driver with version 2016.2 of Vivado, I assume the driver is backward compatible. Is that true?

Thanks

0 Kudos
Contributor
Contributor
5,314 Views
Registered: ‎06-13-2014

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

Hi,

 

Do you mean that you couldn't assert tlast signal because of the ADC data stream is continous? If that, you don't worry because pcie bandwidth is enough high for you to assert it.

 

I have simulated your verilog file, it seems like that the signal timing is not right.

 

捕获.PNG

 

0 Kudos
Visitor jadeanderson
Visitor
5,286 Views
Registered: ‎03-23-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

Yes this is a streaming application where the ADC data is continuous.  I agree that the timing is not correct, I have modified the file and now I have this timing. Which looks correct to me,

Here is the Sim.. 

sim_simple_dma_FF.jpg

however it is still not working correctly when I run a test in hardware.  it hangs the linux host b/c the flow of data seems to never stop. I expected that when you run

 ./dma_from_device -d /dev/xdma0_c2h_0 -f out_1.bin -s 8192 -c 1

 There should be a way for the host to know when the data has arrived.  But in this case it doesn't.  

 

Here is the Hardware ILA.hw_simple_dma_FF.jpg

Also attached is the dmesg log, which was cleared immediately prior to the burst.

 

 

0 Kudos
Visitor jadeanderson
Visitor
5,278 Views
Registered: ‎03-23-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

Files for reference

0 Kudos
Visitor jadeanderson
Visitor
5,238 Views
Registered: ‎03-23-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

Does anyone have any suggestions?

0 Kudos
Moderator
Moderator
5,222 Views
Registered: ‎02-16-2010

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution
Can you check the option to bypass the descriptor?
------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------
0 Kudos
Xilinx Employee
Xilinx Employee
7,854 Views
Registered: ‎05-07-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

HI @jadeanderson

 

Disabling cyclic buffer  might help your use case.
please replace ther xdma-core.c file in AR65444 driver with the the one attached  and then load the driver and run the tests  once

Thanks
Bharath
--------------------------------------------------​--------------------------------------------
Please mark the Answer as "Accept as solution" if information provided addresses your query/concern.
Give Kudos to a post which you think is helpful.
--------------------------------------------------​-------------------------------------------
Visitor jadeanderson
Visitor
5,017 Views
Registered: ‎03-23-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

Hi @nagabhar

Thanks for the help.  Turning off the cyclic buffer works much better in my application.  I was able to test with and without the cyclic transfers, and I see a very dramatic difference in the operation. I was seeing the HOST get flooded with interrupts b/c it was creating 256 Descriptors for each DMA transfer.

  By disabling the cyclic buffering, now when I  run the 'dma_from_device'  I see only 2 interrupts. (descriptor complete, and transfer done)  plus I get the exact amount of data that I asked for.

This is much easier to work with.

 

Thanks

Jade

0 Kudos
Visitor jadeanderson
Visitor
3,049 Views
Registered: ‎03-23-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution
I tried that @venkata, however there is no documentation on how to use the descriptor bypass in the PG195 doc. For example there is a signal called c2h_dsc_byp_ctl[15:0]
The only description is:
"Input Descriptor control to be loaded". That isn't enough information to even get close. If there was more info on how to use this, I would have investigated further.
Thanks
0 Kudos
Visitor jadeanderson
Visitor
2,972 Views
Registered: ‎03-23-2015

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

After further investigation. It looks like there are problems with disabling the cyclic read buffering.

I see entries in dmesg log that shows system memory was corrupted after I do a few c2h transfers.  I am not a software driver guy, so how might one debug this?

 

Thanks

Jade

0 Kudos
2,780 Views
Registered: ‎02-15-2014

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

Hi,

i have the same problem described in the post. Now the driver version in AR65444 is changed and your file wan't compile. Could you update with the last driver or tell me what to change.

 

I have an ILA in design and i see that even if i transfer just 1024 byte by the command 

sudo ./dma_from_device -d /dev/xdma0_c2h_0 -f /media/loki/file.bin -s 1024 -c 1

 

the transfer never stops

 

This is the block diagram

axi stream dma.JPG

 

I think that your solution can also help me

 

Thank you

 

Andrea

0 Kudos
Visitor azbarks
Visitor
2,098 Views
Registered: ‎12-09-2016

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

Has this been added to the newest driver code?  I ask because I don't see a xdma-core.c file in there. 

0 Kudos
Visitor r.sarwar87
Visitor
1,659 Views
Registered: ‎09-17-2018

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

 

@azbarks I attached the file for the driver released on 20180420. It seems to work for me, and should be okay to use. I did not see any corruption upon repeated usage, albeit I have not extensively tested it.

 

it disables the cyclic sgdma transfer when using c2h streaming.

 
 
 
thanks  for the hint, can i ask for an explanation of the bypass control ports? It would be most helpful. 
1,101 Views
Registered: ‎05-18-2018

Re: PCIE DMA subsystem AXI4 Streaming c2h transfers

Jump to solution

Hi @nagabhar

 

I found the latest version "Xilinx_Answer_65444_Linux_Files_rel20180420.zip" still default enable "cyclic buffer transfer" when ST C2H, Would you please:

  1. briefly explain what is "cyclic buffer transfer" ?
  2. What is the advantage of it ?

Thank you.

 

0 Kudos