10-28-2019 05:47 AM
I'm implementing the PCIe EP example design, in the PL of an UltraScale+ MPSoC Zynq. I've tried several configurations, up to x4 gen3, but they all lead to the same error: link training does not complete.
I can see that the 100MHz ref-clk and PERSTn are behaving as expected. The design gets to Configuration state 8 in the LLSSM, but never any further (see attachments).
Can anyone suggest a line of investigation?
Thanks in advance,
10-28-2019 01:31 PM
You're on the right path by using the PCIe JTAG debugger and sending the screenshots. You've tried gen2x1, lower speeds by making a new bitfile with the IP configured for the different rates? Do you have the sys_clk and sys_clk_gt setup and configured correctly according to the line rate? What are the capabilities of the RP and PCIe slot/channel you're plugging into? Can you add in-system IBERT to the design (PG213 p.310)?
Also this post and AR might help:
10-28-2019 02:38 PM
Thanks for replying.
Yes, I've tried other configurations - right down to gen1 x1 - but this didn't help. I could try this again to be sure, but I expected the RP to down-train to whatever speed it could support? So if the channel could only support gen2 speeds (for example), then that's what I would get?
The link-partner (RP) is a PC, which is capable of gen3 speeds. We've seen this channel working before, with a gen2 EP.
I've added an IBERT into the FPGA design, and run local loopback at 8Gbps, and everything looks good. When I run an IBERT sweep with no link established, I see no eye (as expected).
About the clocks, I'm using the example design (screenshots of GUI attached). I'm using the 100Mz reference clock to derive all clocks, so I'm not reliant on any other clock. I allow the core to generate my user_clock for me.
Would you recommend a different clocking scheme?
10-29-2019 06:49 AM
Using the 100MHz PCIe clock should be fine. I notice your PCIE block location is X0Y0 and your GT's are at 224. The PCIe block should be at X1Y0 to be on the right side of the die next to the GTH. Refer to the bank diagram for your package in UG1075.
10-30-2019 02:54 AM
I made the change to X1Y0, but the change didn't help. Still the link was not established.
Any other ideas?
10-30-2019 04:34 AM
...and just to confirm, I have assiged the PCIe lanes 3210 to GTH lanes 0123.
That's no problem, is it? Lane-order reversal is automatically included, right?
10-30-2019 08:33 AM
...and can I confirm that each GTH differential pair can be routed with any polarity, leaving the receiver to invert if necessary?
I ask as our working gen2 platform has its RX pairs inverted.
11-04-2019 09:41 AM
Been there, done that! Seems you found the problem. I will bet you found your problem in the auto-endianism design spec for PCIe lane determantation. I believe the concept/spec is flawed. I ran across a board manufacturer who swizzled their lanes. This precluded anything but full lane occupancy for anything to work. For example, your 2 lanes might not get detected on a 4 lane interface.
You can test for this by building x1 one designs that individually test each lane.
11-04-2019 02:02 PM
It was related to that.
Viviado will allow the re-assignment of lanes wrt pin-pairs, by modification of the constraints or the Veriog in the exmaple design. The implementation passes, and a bitfile is generated.
...except the bitfile doesn't work, unless the option is enabled NOT to fix lane/pair mapping when generating the core in the example design.