cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
faku99
Visitor
Visitor
557 Views
Registered: ‎03-19-2020

Second C2H DMA doesn't assert tready signal

First of all, here is the design I'm using:

XDMA designXDMA design

The first DMA channel (C2H_0) initialization is done correctly. After allocating memory, descriptors and requesting a transfer, the `tready` signal is asserted and I can run multiple transfers without any issues.

But when the second DMA channel (C2H_1) initialization is (successfully) done, the `tready` signal simply stays at 0. It never takes the value 1. I already checked if there could be some non-wanted `tlast` or `tvalid` before initialization but that's not the case.

Below, find a dump of both DMA channels after initialization:

[ 310.820955] selftest_qrng:sgt_alloc_with_pages: 0-C2H0-ST: Successfully allocated PCI coherent memory - phys: 0x3C000000, virt: 0xAC000000, size: 0x20000
[ 310.840149] selftest_qrng:engine_reg_dump: 0-C2H0-ST: ioread32(0xe6281406) = 0x1fc18006 (id).
[ 310.848698] selftest_qrng:engine_reg_dump: 0-C2H0-ST: ioread32(0x93a33b61) = 0x00000001 (status).
[ 310.857574] selftest_qrng:engine_reg_dump: 0-C2H0-ST: ioread32(0x159b0e0e) = 0x04f83e19 (control)
[ 310.866467] selftest_qrng:engine_reg_dump: 0-C2H0-ST: ioread32(0x0d4507f3) = 0x3c080000 (first_desc_lo)
[ 310.875874] selftest_qrng:engine_reg_dump: 0-C2H0-ST: ioread32(0x735f40c1) = 0x00000000 (first_desc_hi)
[ 310.885291] selftest_qrng:engine_reg_dump: 0-C2H0-ST: ioread32(0x28744a12) = 0x0000000f (first_desc_adjacent).
[ 310.895304] selftest_qrng:engine_reg_dump: 0-C2H0-ST: ioread32(0x1fe7ae6e) = 0x00000000 (completed_desc_count).
[ 310.905413] selftest_qrng:engine_reg_dump: 0-C2H0-ST: ioread32(0x3730dec6) = 0x00f83e18 (interrupt_enable_mask)
[ 310.915873] selftest_qrng:sgt_alloc_with_pages: 0-C2H1-ST: Successfully allocated PCI coherent memory - phys: 0x3C020000, virt: 0xAC020000, size: 0x20000
[ 310.938338] selftest_qrng:engine_reg_dump: 0-C2H1-ST: ioread32(0x7e61b4f9) = 0x1fc18106 (id).
[ 310.946865] selftest_qrng:engine_reg_dump: 0-C2H1-ST: ioread32(0xe4974543) = 0x00000001 (status).
[ 310.955760] selftest_qrng:engine_reg_dump: 0-C2H1-ST: ioread32(0x49369d06) = 0x04f83e19 (control)
[ 310.964648] selftest_qrng:engine_reg_dump: 0-C2H1-ST: ioread32(0xee13876c) = 0x3c0a0000 (first_desc_lo)
[ 310.974061] selftest_qrng:engine_reg_dump: 0-C2H1-ST: ioread32(0xe2796730) = 0x00000000 (first_desc_hi)
[ 310.983474] selftest_qrng:engine_reg_dump: 0-C2H1-ST: ioread32(0x393b21b4) = 0x0000000f (first_desc_adjacent).
[ 310.993496] selftest_qrng:engine_reg_dump: 0-C2H1-ST: ioread32(0xf06c42d3) = 0x00000000 (completed_desc_count).
[ 311.003595] selftest_qrng:engine_reg_dump: 0-C2H1-ST: ioread32(0xe15fe714) = 0x00f83e18 (interrupt_enable_mask)

The registers dump looks fine to me.

For the sake of curiosity, I swapped both channels and surprisingly, the channel C2H_0 works but C2H_1 doesn't.

Thus, the problem comes from the driver, and I suspect an issue with some index somewhere.

Don't hesitate if you need some more information.

Lucas

7 Replies
faku99
Visitor
Visitor
506 Views
Registered: ‎03-19-2020

By searching on the internet for people that got the same issue, I found this interesting thread: https://forums.xilinx.com/t5/PCIe-and-CPM/XDMA-C2H-bug-XC7A35T-vs-XC7A200T/td-p/1143941.

 

It seems that there is a bug in the DMA/Bridge Subsystem for PCI Express IP, specially in the 4.2 version when targeting the XC7A200T part. Surprisingly (?), I happen to use the exact same Vivado version (2019.2), the exact same IP version (4.1) and the exact same target (XC7A200T).

 

How can I report it directly to Xilinx? To avoid that this topic will also be lost among the many others...

0 Kudos
faku99
Visitor
Visitor
470 Views
Registered: ‎03-19-2020

Also, I enabled debug logging on both C2H engines and both produce the same output (except for the addresses, of course). You can find both logs attached.

0 Kudos
liy
Xilinx Employee
Xilinx Employee
268 Views
Registered: ‎08-02-2007

Are you able to reproduce this issue with the example design?

 

------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------
0 Kudos
faku99
Visitor
Visitor
228 Views
Registered: ‎03-19-2020

Yes, we are able to reproduce this issue with the example design.

 

Additionally, we did further tests on our side and we noticed something really strange: we changed the number of H2C channels from 1 to 2, but both H2C channels are left unconnected. When doing this, both C2H channels don't assert the TREADY signal.

So, to summarize the tests performed and the results observed (H2C channels are left unconnected):

  • 2 C2H channels, 1 H2C channel: channel C2H_0 asserts TREADY, C2H_1 doesn't
  • 2 C2H channels, 2 H2C channels: both C2H channels don't assert TREADY

 

It seems that the Number of H2C channels parameter interferes with the TREADY signal behavior of the C2H channels...

0 Kudos
venkata
Moderator
Moderator
173 Views
Registered: ‎02-16-2010

Hi @faku99 ,

I tried to enable 2 C2H channes, 2 H2C channels with AXI4-Stream interface. I generated IP example design and updated the test bench to do the DMA transfer on both the channels. I get the result shown below.

venkata_0-1622588117708.png

I have sent the project archive to you through EZmove. Please check dma_stream0 test in sample_tests.vh file in imports directory and confirm if you are doing the same steps with your design.

------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution
------------------------------------------------------------------------------
faku99
Visitor
Visitor
63 Views
Registered: ‎03-19-2020

Hi @venkata,

Other projects were taking a lot of my time but I've finally been able to test out the project you sent me via EZmove.

I had to change it a bit (added an ILA core and changed the target to XC7A200T) but it works.

So, I rolled back my project to the previous version and programmed my FPGA with it. Same error, TREADY is not asserted by C2H_1...

So, I looked for the differences between your code and mine, but I found none relevant. At this point, the only differences are the IP parameters.

I opened my project's block design, set the XDMA parameters to the exact same ones set in the project you sent me, and... It works! Hooray

I have not yet isolated the parameter that causes the problem but I might soon.

Here is the list of the parameters changed (left not working, right working):

  • Basic tab
    • Mode: Advanced > Basic
    • Maximum Link Speed: 5 GT/s > 2.5 GT/s
  • PCIe: MISC tab
    • Configuration Extended Interface: Disabled > Enabled
  • PCIe: DMA tab
    • # Request IDs for Read/Write channel: 2 > 32/16

 

0 Kudos
faku99
Visitor
Visitor
33 Views
Registered: ‎03-19-2020

Follow-up: It is the Number of Request IDs for Read channel parameter that causes the issue. When set to 2, TREADY misbehaves. When set to 4 or more, it behaves normally.

0 Kudos