cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
blynskey_thinci
Observer
Observer
1,091 Views
Registered: ‎07-25-2018

UltraScale+ PCIe EP example design - problem in Link Training (Vivado 2019.1)

Jump to solution

Hi,

I'm implementing the PCIe EP example design, in the PL of an UltraScale+ MPSoC Zynq. I've tried several configurations, up to x4 gen3, but they all lead to the same error: link training does not complete.

I can see that the 100MHz ref-clk and PERSTn are behaving as expected. The design gets to Configuration state 8 in the LLSSM, but never any further (see attachments).

 

Can anyone suggest a line of investigation?

 

Thanks in advance,

 

 - Brendan

output_draw_ltssm.png
output_draw_reset.png
output_draw_rxdet.png
ILA_LTSSM.png
0 Kudos
Reply
1 Solution

Accepted Solutions
blynskey_thinci
Observer
Observer
873 Views
Registered: ‎07-25-2018

Have now fixed tis - thanks for the help!

View solution in original post

0 Kudos
Reply
9 Replies
jbeckwi
Xilinx Employee
Xilinx Employee
1,048 Views
Registered: ‎08-30-2011

You're on the right path by using the PCIe JTAG debugger and sending the screenshots.  You've tried gen2x1, lower speeds by making a new bitfile with the IP configured for the different rates?  Do you have the sys_clk and sys_clk_gt setup and configured correctly according to the line rate?  What are the capabilities of the RP and PCIe slot/channel you're plugging into?  Can you add in-system IBERT to the design (PG213 p.310)?

Also this post and AR might help:

https://forums.xilinx.com/t5/Design-and-Debug-Techniques-Blog/Debugging-PCIe-Link-Training-Issues-Part-One/ba-p/967123

 

Jon

0 Kudos
Reply
blynskey_thinci
Observer
Observer
1,036 Views
Registered: ‎07-25-2018

Hi,

Thanks for replying.

Yes, I've tried other configurations - right down to gen1 x1 - but this didn't help. I could try this again to be sure, but I expected the RP to down-train to whatever speed it could support? So if the channel could only support gen2 speeds (for example), then that's what I would get?

The link-partner (RP) is a PC, which is capable of gen3 speeds. We've seen this channel working before, with a gen2 EP.

I've added an IBERT into the FPGA design, and run local loopback at 8Gbps, and everything looks good. When I run an IBERT sweep with no link established, I see no eye (as expected).

About the clocks, I'm using the example design (screenshots of GUI attached). I'm using the 100Mz reference clock to derive all clocks, so I'm not reliant on any other clock. I allow the core to generate my user_clock for me.

Would you recommend a different clocking scheme?

Best regards,

- Brendan

USp_PCIe_integ_block_GUI_001.png
USp_PCIe_integ_block_GUI_002.png
USp_PCIe_integ_block_GUI_003.png
USp_PCIe_integ_block_GUI_004.png
0 Kudos
Reply
jbeckwi
Xilinx Employee
Xilinx Employee
987 Views
Registered: ‎08-30-2011

Hi Brendan,

Using the 100MHz PCIe clock should be fine.  I notice your PCIE block location is X0Y0 and your GT's are at 224.  The PCIe block should be at X1Y0 to be on the right side of the die next to the GTH.  Refer to the bank diagram for your package in UG1075.

Thanks,

Jon

0 Kudos
Reply
blynskey_thinci
Observer
Observer
954 Views
Registered: ‎07-25-2018

Hi,

 

I made the change to X1Y0, but the change didn't help. Still the link was not established.

 

Any other ideas?

 

Best regards,

 

 - Brendan

0 Kudos
Reply
blynskey_thinci
Observer
Observer
942 Views
Registered: ‎07-25-2018

...and just to confirm, I have assiged the PCIe lanes 3210 to GTH lanes 0123.

That's no problem, is it? Lane-order reversal is automatically included, right?

 

 - Bren

0 Kudos
Reply
blynskey_thinci
Observer
Observer
907 Views
Registered: ‎07-25-2018

...and can I confirm that each GTH differential pair can be routed with any polarity, leaving the receiver to invert if necessary?

 

I ask as our working gen2 platform has its RX pairs inverted.

 

Best regards,

 

 - Brendan

0 Kudos
Reply
blynskey_thinci
Observer
Observer
874 Views
Registered: ‎07-25-2018

Have now fixed tis - thanks for the help!

View solution in original post

0 Kudos
Reply
thomasmullin
Observer
Observer
826 Views
Registered: ‎11-20-2013

Been there, done that! Seems you found the problem.  I will bet you found your problem in the auto-endianism design spec for PCIe lane determantation.  I believe the concept/spec is flawed. I ran across a board manufacturer who swizzled their lanes.  This precluded anything but full lane occupancy for anything to work.  For example, your 2 lanes might not get detected on a 4 lane interface.

You can test for this by building x1 one designs that individually test each lane.  

0 Kudos
Reply
blynskey_thinci
Observer
Observer
811 Views
Registered: ‎07-25-2018

It was related to that.

Viviado will allow the re-assignment of lanes wrt pin-pairs, by modification of the constraints or the Veriog in the exmaple design. The implementation passes, and a bitfile is generated.

...except the bitfile doesn't work, unless the option is enabled NOT to fix lane/pair mapping when generating the core in the example design.

 - Bren

0 Kudos
Reply