DMA/Bridge 3.0 C2H-ST transaction returns -1 or 29 bytes when source is custom RTL block

[darktyr]

HI,

Basically I am trying to understand how the DMA/Bridge functions and what I have to do to interface to the C2H AXI-Stream port. So I put together a block design with the DMA/Bridge, couple of AXI-Lite peripherals (BRAM controller, timers, I2C), two loopback streaming FIFO and my custom RTL block wired into a FIFO then to the AXI-Stream switch. I have the AXI-Stream switches configured for user configuration. The attached PDF is the block diagram in PDF form.

I believe I have my stream generator correct and then correctly configured to work with the DMA engine.

tkeep is tied high, tlast is high for one clock cycle on the tvalid after 128 beats (128 beats * 32 bytes = 4096 = transfer size in console) has passed. Below is a xsim screenshot showing the timing after 256 beats (8192 byte transfer).

xsim_axis_gen_type0.PNG

 

The idea with the data generator is to slow the data rate down to what I am expecting, 125MB/s to 250MB/s and to keep track of samples.

That hopefully gives you the idea on the firmware side. Software side, I started using the the new Linux driver and enabled debugging which worked instantly and great for the loop back tests. I have a script the configures the data routers before running running the dma_to_device and dma_from_device. This works up to the size of the largest FIFO (512kB). So I don't think it has anything to do with the driver or software I have written. When I switch over to my axi stream generator I have all of the parts form the dma_to_device and dma_from_device in a script that I call. The script configures the data routers, simulator, enables the simulator then asks for data. I ensure that the number of bytes I am asking for matches the number of bytes between tlast pulses.

Using the new Linux driver I get an immediate return value of -1 from the file read() call (my script and dma_from_device give the same thing). Also the driver is in poll_mode, using interrupts would hang. Going to the old Linux driver it changes a little bit in that I will get 29 bytes.... I am not sure what has changed in the background of the driver.

dma_from_device_console_output.png

That screenshot is with the old Linux driver.

Another difference between the drivers is the amount of dmesg output when debugging is enabled. The new driver output nothing when running dma_from_device. The old driver would actually say:

xmda:complete_cyclic: 0-C2H-ST result[0] length 0x1d

So it seems the DMA engine thinks it finished the transaction when it is being polled. When the driver is installed to use interrupts it just hangs. I don't know enough why it would do that. I am just looking to figure out what I am doing wrong and where.

New driver : Xilinx_Answer_65444_Linux_Files_rel20180420.zip

old driver: Xilinx_Answer_65444_Linux_Files.zip

Other info:

Vivado 2016.4 on Windows 10

Ultrascale 115 card running in an Ubuntu 16.04 computer

DMA/Bridge 3.0

Gen2.0 x8 lane endpoint

 
My References:

https://www.xilinx.com/support/documentation/ip_documentation/xdma/v3_0/pg195-pcie-dma.pdf

https://www.xilinx.com/support/answers/65444.html

https://forums.xilinx.com/t5/PCI-Express/DMA-Subsystem-for-PCIE-axi-stream-C2H-transfers/td-p/746740
https://forums.xilinx.com/t5/PCI-Express/PCIE-DMA-subsystem-AXI4-Streaming-c2h-transfers/td-p/791701/highlight/true/page/2
https://forums.xilinx.com/t5/PCI-Express/DMA-Bridge-Subsystem-for-PCIe-AXI-Stream-c2h-interface/m-p/815172#M9553
https://forums.xilinx.com/t5/PCI-Express/PCIe-streaming/td-p/812679

I feel like I am missing something simple. I am going to try and put the FIFO's into packet mode and see if that makes a difference. I am not sure why it would make a difference though.

Dark Tyr

[darktyr · Accepted solution]

My problem has been fixed and was related to using tkeep versus tstrb. The DMA/Bridge subsystem product guide never mentions tstrb and instead only mentions tkeep (v3.0 and v4.0). This does contradict what is stated in the AXI Reference Guide:

https://www.xilinx.com/support/documentation/ip_documentation/ug761_axi_reference_guide.pdf

Which states that tkeep should be absent and that tstrb is optional/used. It may be more related to how the Xilinx IP blocks handle tstrb and tkeep (Xilinx FIFO and Xilinx Stream Switch).

The reason for the odd number of bytes being returned was that it was getting tlast and tvalid but never any valid bytes.

I now use tstrb to interface to the Xilinx FIFO instead of tkeep and leave it on auto as well as the stream switch on auto.  

Dark Tyr

[ndnsoulja]

@darktyr I'm doing something similar to what you've accomplished and want to thank you for this post and followup. 

I know this is old and hopefully you're still active on the forums. but could you example how you did this: " I have a script the configures the data routers before running running the dma_to_device and dma_from_device."