09-05-2017 12:36 PM
I am trying use the Xilinx PCIE DMA Subsytem with my KCU105 board. The problem I have is that when I use the Card2Host AXI streaming channel, I get more data than I expect.
I haven’t found a way to get data back to the host without generating an AXI TLAST pulse. So I have a counter that counts the number of 32byte AXI4-S Data transfers, then it generates a pulse on the Tlast input of the DMA C2H port. I must be doing this incorrectly b/c when I send the pulse, the DMA should sense that the transfer is finished, and then stop the request for data.
I have data flowing from the ADC and it is valid every clock cycle, so I need to get that into the HOST and do so without losing samples. I have plenty of bandwidth on the FPGA to Host.
I have not figured out how to do this cleanly. I get the following:
I have my transfer size and my internal FPGA counter set exactly the same, so for a 131072 byte transfer I should get 1 tlast pulse.
./dma_from_device -d /dev/xdma0_c2h_0 -f output_data.bin -s 131072 -c 1
sscanf() = 1, value = 0x00020000
sscanf() = 1, value = 0x00000001
device = /dev/xdma0_c2h_0, address = 0x00000000, size = 0x00020000, offset = 0x00000000, count = 1
host memory buffer = 0x7f9639ad3000
CLOCK_MONOTONIC reports 0.000112370 seconds (total) for last transfer of 131072 bytes
Done
Tail OF THE DMESG LOG on Linux HOST PC
[105773.751115] ch_irq = 0x00000002
[105773.751126] ch_irq = 0x00000002
[105773.751136] ch_irq = 0x00000002
[105773.751146] ch_irq = 0x00000002
[105773.751157] ch_irq = 0x00000002
[105773.751167] ch_irq = 0x00000002
[105773.751177] ch_irq = 0x00000002
[105773.751187] ch_irq = 0x00000002
[105773.751197] ch_irq = 0x00000002
[105773.751208] ENTERING char_sgdma_close,total read=370
[105773.751209] ch_irq = 0x00000002
[105773.751218] ch_irq = 0x00000002
The ILA in my design shows that the c2h_s_axis_tready line is ready for more data, I thought it should go low and stay low after the transfer was complete. The main problem is that I am blindly generating the TLAST signal based on a hard coded transfer size and I can only request that size in the host. There must be a better way to send data samples from FPGA to Host with this C2H streaming interface. If not, then at the least, I need clean way to reset the TLAST counter at the beginning of a transfer. If I knew the number of bytes requested, then I could load the counter value at the beginning of the transfer and then send the tlast pulse at the exact boundary.
Thanks
09-18-2017 04:09 AM
Disabling cyclic buffer might help your use case.
please replace ther xdma-core.c file in AR65444 driver with the the one attached and then load the driver and run the tests once
09-05-2017 04:51 PM
09-06-2017 08:56 AM
I tried doing this, but the problem that I face now is that there are so many channel Interrupts that it locks up the host. So can you point me to a simple AXI streaming generator that I can try ? I built my own, but I don't know how to control the fact that I always have valid data since my application is ADC > FIFO > PCIE_DMA
I was going to mimic this with a simple verliog model.
drive t_data port with a counter
drive t_valid and t_last based on the rising edge of the t_ready from the host.
See attached file.
Also, can I am trying to use the latest Linux driver with version 2016.2 of Vivado, I assume the driver is backward compatible. Is that true?
Thanks
09-07-2017 11:24 PM
Hi,
Do you mean that you couldn't assert tlast signal because of the ADC data stream is continous? If that, you don't worry because pcie bandwidth is enough high for you to assert it.
I have simulated your verilog file, it seems like that the signal timing is not right.
09-10-2017 01:45 PM
Yes this is a streaming application where the ADC data is continuous. I agree that the timing is not correct, I have modified the file and now I have this timing. Which looks correct to me,
Here is the Sim..
however it is still not working correctly when I run a test in hardware. it hangs the linux host b/c the flow of data seems to never stop. I expected that when you run
./dma_from_device -d /dev/xdma0_c2h_0 -f out_1.bin -s 8192 -c 1
There should be a way for the host to know when the data has arrived. But in this case it doesn't.
Here is the Hardware ILA.
Also attached is the dmesg log, which was cleared immediately prior to the burst.
09-10-2017 05:56 PM
09-14-2017 09:08 AM
Does anyone have any suggestions?
09-15-2017 02:30 PM
09-18-2017 04:09 AM
Disabling cyclic buffer might help your use case.
please replace ther xdma-core.c file in AR65444 driver with the the one attached and then load the driver and run the tests once
09-28-2017 07:05 PM
Hi @nagabhar,
Thanks for the help. Turning off the cyclic buffer works much better in my application. I was able to test with and without the cyclic transfers, and I see a very dramatic difference in the operation. I was seeing the HOST get flooded with interrupts b/c it was creating 256 Descriptors for each DMA transfer.
By disabling the cyclic buffering, now when I run the 'dma_from_device' I see only 2 interrupts. (descriptor complete, and transfer done) plus I get the exact amount of data that I asked for.
This is much easier to work with.
Thanks
Jade
09-28-2017 07:12 PM
10-31-2017 11:03 AM
After further investigation. It looks like there are problems with disabling the cyclic read buffering.
I see entries in dmesg log that shows system memory was corrupted after I do a few c2h transfers. I am not a software driver guy, so how might one debug this?
Thanks
Jade
12-18-2017 09:41 AM
Hi,
i have the same problem described in the post. Now the driver version in AR65444 is changed and your file wan't compile. Could you update with the last driver or tell me what to change.
I have an ILA in design and i see that even if i transfer just 1024 byte by the command
sudo ./dma_from_device -d /dev/xdma0_c2h_0 -f /media/loki/file.bin -s 1024 -c 1
the transfer never stops
This is the block diagram
I think that your solution can also help me
Thank you
Andrea
07-23-2018 12:48 PM
Has this been added to the newest driver code? I ask because I don't see a xdma-core.c file in there.
11-27-2018 03:16 PM - edited 11-28-2018 08:55 AM
@azbarks I attached the file for the driver released on 20180420. It seems to work for me, and should be okay to use. I did not see any corruption upon repeated usage, albeit I have not extensively tested it.
it disables the cyclic sgdma transfer when using c2h streaming.
01-21-2019 08:55 AM
Hi @nagabhar
I found the latest version "Xilinx_Answer_65444_Linux_Files_rel20180420.zip" still default enable "cyclic buffer transfer" when ST C2H, Would you please:
Thank you.