cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
stefangriebel
Observer
Observer
1,029 Views
Registered: ‎11-09-2011

SRIO port_init failure, never get to link_init

Jump to solution

I am using the following tool versions and FPGAs

  • Vivado 2019.1 and 2019.2
  • Kintex Ultrascales, ku060
  • SRIO Gen2 Endpoint v4.1
    • 156.25 MHz refclk
    • 3.125 GBPS links
    • 4 lanes

I have two separate boards, both with ku060s. I am having this problem on all quads, but I will limit the discussion to 1 quad here. The two boards are cabled together with custom cables, and I have verified this is not a hardware / SI problem by using standalone IBERT builds in both FPGAs. I achieve link on all lanes, all quads using PRBS 7-bit in a variety of configurations: no loopback, far-end loopback on one side, vice-versa. Eye diagrams look good.

nice_iberts.png

However, when I have two SRIO cores in either FPGA, I have not found a start-up or reset sequence that allows all 4-lanes to achieve link-init. I have tried various TXDIFF settings from max to min amplitude, as well as enabling LPM mode.

I inserted an ILA and have made the following observations: (full graphic also uploaded as attachment)

port_init_debug.png

What could be causing the lanes 1 and 3 to drop out completely to zero? What signals should I look at next for further debug?

0 Kudos
1 Solution

Accepted Solutions
guozhenp
Xilinx Employee
Xilinx Employee
547 Views
Registered: ‎05-01-2013

The box is for the error after link up. It doesn't help.

The initialization follows SRIO spec. You've to make 2 sides reset at the same time to get 4x link.

I have a reference code to improve the initialization as follows. You can have a try.

 

// if link_initialized down force link initialization
parameter siso_shift_init = 8;
reg [siso_shift_init-1:0] sShiftReg_init = {siso_shift_init{1'b1}};

always @(posedge freerun_clk)
if ( !mode_1x && phy_debug[34] && link_initialized)
sShiftReg_init <= 0;
else if (!link_initialized && phy_debug[34])
sShiftReg_init[7] <= 1;
else
sShiftReg_init <=

{0, sShiftReg_init[siso_shift_init-1:1]}

;

assign force_reinit = sShiftReg_init[0];

View solution in original post

14 Replies
guozhenp
Xilinx Employee
Xilinx Employee
935 Views
Registered: ‎05-01-2013

Please try GT PMA near end loopback first. Can SRIO Gen2 get link_initialized?

0 Kudos
stefangriebel
Observer
Observer
927 Views
Registered: ‎11-09-2011

Yes, both sides get link_initialized in near-end PMA loopback mode every time. Both sides can also get link_initialized in an "external loopback" mode when the other side contains only IBERT cores placed in far-end PMA loopback. As soon as both sides have SRIO cores, I get the problem described above.

Also worth noting, when one side has an SRIO core placed in far-end PMA loopback, and the other side has an IBERT core sending PRBS, only lanes 0 and 2 will ever achieve link. It's as if the SRIO core inhibits lanes 1 and 3 even when configured in far-end PMA loopback.

0 Kudos
guozhenp
Xilinx Employee
Xilinx Employee
910 Views
Registered: ‎05-01-2013

If you try to reset both sides SRIO at the same time, can they link up?

When SRIO fails to link with 4x, it will train down to 1x automatically. So it's normal to see lanes 1 and 3 not working at that time.

0 Kudos
stefangriebel
Observer
Observer
889 Views
Registered: ‎11-09-2011

I do frequently get a trained-down link in mode_1x. And very, very rarely, I will get a 4x link with "disperr" and "notintable" low. This happens about 1 per 100 tries, so it can be done, but I can't get the links up reliably.

0 Kudos
guozhenp
Xilinx Employee
Xilinx Employee
858 Views
Registered: ‎05-01-2013

PG007 says,

Special care must be used when
resetting the SRIO Gen2 Endpoint device. Both link partners should be reset together to
guarantee ackID alignment. It is recommended that resets between link partners be
overlapped to reduce the occurrence of lost packets and control symbols. One way to
implement this is to handshake the core resets. Resets received from the link partner are
communicated to the user design with the assertion of phy_rcvd_link_reset from the
core. The sys_rst signal should be asserted upon receipt of a link reset. Depending on the
implementation, a reset can also be signaled to the user application in response to the
assertion of phy_rcvd_link_reset. To send a reset request to the link partner, assert the
phy_link_reset signal until the port_initialized output goes Low. At this time, sys_rst
should be asserted to the reset reference design, completing the handshake.

0 Kudos
stefangriebel
Observer
Observer
814 Views
Registered: ‎11-09-2011

Using the phy_rcvd_link_reset handshaking described above, I can get to link_init successfully about 1 in 5 tries. However, I frequently get stuck and still have to issue a sys_rst to either one or both of the link partners to get back to a stable state, then try again. The 4 out of 5 tries that fail look like the condition described in the OP.

0 Kudos
guozhenp
Xilinx Employee
Xilinx Employee
790 Views
Registered: ‎05-01-2013

When you say "get stuck in 4 out of 5 tries", do they have link_initialzed with mode_1x or 4 lanes can't get port_initialized? In both ends? (these 2 SRIO may have the differnt failure)

There're multiple issues in the SRIO 4x initialization. We need more details on the failure. While all of them can be fixed if you can make 2 sides resets released at the exact same time.

0 Kudos
stefangriebel
Observer
Observer
659 Views
Registered: ‎11-09-2011

When I say "stuck 4 out of 5 tries," the cores are usually reporting both link_initialized and port_initialized on both ends, but are in mode_1x.

I have started investigating this again, and one thing I've noticed is that using a single txoutclk from the "master" core, anytime that core undergoes a reset, all output clocks are lost, and any other cores will lose link. Is there a way around this without using a separate txoutclk for every core?

0 Kudos
guozhenp
Xilinx Employee
Xilinx Employee
617 Views
Registered: ‎05-01-2013

It's expected.

When the master does reset, the clock shared with other slaves will be lost. So all the salves also need to do reset.

0 Kudos
stefangriebel
Observer
Observer
590 Views
Registered: ‎11-09-2011

OK, good to know about the clocks. 

Why does a link get stuck in mode_1x? Is there a way to force the core to drop link_init and keep trying to renegotiate at the intended link speed? I currently have the "infinite" box checked in my GUI config, but setting it to 3 vs. 6 v.s infinite doesn't seem to make a difference.

infinite_retries.png
0 Kudos
guozhenp
Xilinx Employee
Xilinx Employee
548 Views
Registered: ‎05-01-2013

The box is for the error after link up. It doesn't help.

The initialization follows SRIO spec. You've to make 2 sides reset at the same time to get 4x link.

I have a reference code to improve the initialization as follows. You can have a try.

 

// if link_initialized down force link initialization
parameter siso_shift_init = 8;
reg [siso_shift_init-1:0] sShiftReg_init = {siso_shift_init{1'b1}};

always @(posedge freerun_clk)
if ( !mode_1x && phy_debug[34] && link_initialized)
sShiftReg_init <= 0;
else if (!link_initialized && phy_debug[34])
sShiftReg_init[7] <= 1;
else
sShiftReg_init <=

{0, sShiftReg_init[siso_shift_init-1:1]}

;

assign force_reinit = sShiftReg_init[0];

View solution in original post

stefangriebel
Observer
Observer
522 Views
Registered: ‎11-09-2011

OK, I will give this a try. Should I connect the force_reinit in your code directly to the SRIO core (my clocks and reset are external to the core) or should I connect that force_reinit to the srio_rst.v module which has the reset FSM etc and outputs a controlled_force_reinit currently connected to the core.

0 Kudos
guozhenp
Xilinx Employee
Xilinx Employee
475 Views
Registered: ‎05-01-2013

You can just try both ways.

I prefer the FSM input.

0 Kudos
stefangriebel
Observer
Observer
417 Views
Registered: ‎11-09-2011

This appears to be the Golden Bullet for my issues, specifically Xilinx SRIO Core <---> Xilinx SRIO Core in separate FPGAs! The auto force-init does break my other SRIO links to non-Xilinx devices, so I simply added an enable bit to the code snippet provided above.

Thanks,
Stefan

0 Kudos