05-23-2018 01:29 PM
I have developed a custom interconnection system for simplex communication across FMCs connectors (VC707 as target board). Both FMC1 and FMC2 are plugged on custom boards, connected together through an hi-speed cable (100Ohm diff.). The cable can have two different lengths (4 or 20 cm), both carrying 8 lanes running at 8Gbps.
Each board (both on TX and RX side), cleans an 80MHz clock coming from a PLL ( OSC -> PLL -> OBUFDS -> ODDR -> CONNECTOR) through a jitter cleaner and sends it as a REFCLK of the quad (each quad has its 80MHz ref_clk).
I have tested the 4cm cable system with IBERT on the same FPGA: all the lanes are locked with 0 errors (running for 1 day) and the sweeps return the best results: Open Area: 3072; Open UI %: 66,72%. I am also able to correctly transmit data with Aurora 64b/66b without error and with a full bandwidth.
When I test the 20cm cable-based system with IBERT, I noticed that lanes lock is random and is harder to get for slowing changing signal patterns. In particular, sweeps return best results of 100% open UI (2000 to 11000 open area) for fast clock pattern on both TX and RX, a 100% open UI (2000 to 11000 open area) for fast clock pattern on RX and slow clock patterns for TX, a 55% open UI for slow clock pattern on both TX and RX and a 0% of open area for a 31b PRBS pattern on both TX and RX.
However, even with 31b PRBS pattern, all lanes are randomly (i.e depending on FPGA configuration runs and RX-resets) locked with 0 errors, making me thinking of wrong PMA/PCS settings more than a signal integrity issue.
As a reference, I have attached IBERT links detection after an FPGA configuration (Links 1, 2, 5, 6, 7 up) and after an RX reset (all links down).
Does anybody have any suggestion?
05-23-2018 01:40 PM
Now is the time to start doing parameter sweeps on the TX and RX parameters. It could be that you need to raise the differential TX voltage (very likely even), and possibly modify the preamble/postamble parameters to get a better waveform at the RX. And on the RX side, you can try enabling/disabling the DFE, along with filter parameters for the linear equalizers.. It's a little complicated to figure out which parameter to try first, but if you give it a go and watch the results carefully, you can do a gradient descent to the parameters which produce the maximum eye opening.
05-26-2018 07:28 AM
@jmcclusk, thanks for your reply.
I have passed the last days doing parameter sweeps on GXT links. Best results are reached with low RXTERM and TXDIFFSWING, independently on TXPOST AND TXPRE. In particular, eyes are always 100% opened with low RXTERM and TXDIFFSWING, and goes to 0% when RXTERM = 1100 mV and TXDIFFSWING = 1119 mV.
However, even if I set the links with parameters that return a 100% opened eye, I can't get locked links. The only way I have to lock all links is to configure them firstly with a fast clock pattern on both RX and TX. Then, once they are locked with a fast clock pattern, they remain locked and returns 0 errors even with the other patterns.
Also trying to set the best parameter sweep results on the Aurora design (fully working with the 4cm cable-based system) I am not able to have channel up and lane up triggered on the RX side.
I have also identified some parameters on gtxe2_i that can be used to help with this problem, such as CHAN_BOND_SEQ_LEN, RXBUF_ADDR_MODE, RXBUF_RESET_ON_COMMAALIGN, RXSLIDE_AUTO_WAIT, but even if I change them on the Aurora design, the result is the same.
05-26-2018 12:39 PM - edited 05-26-2018 12:45 PM
It's possible you have so much signal distortion on the 20cm link that 8gps won't work... so let me ask this question.. can you get PRBS31 to lock at 4 gbps? Your goal should be to get PRBS31 locking at 8 Gbps, but you may have an hardware issue that prevents this. Experimenting at a lower frequency may help you pinpoint the problem.
This document may help:
05-28-2018 06:22 AM - edited 05-28-2018 06:51 AM
@jmccluskthanks again for your support.
I have tried to run Aurora with 4Gbps but the RX side still doesn't trigger channel_up and lanes_up. IBERT is able to lock slightly easier with 4Gbps but there are still problems in gettings all lanes to lock together.
As I mentioned in the previous post, even at 8Gbps, once lanes are locked they exhibit a perfectly stable communication channel, producing 0 Errors for hours whatever TX-PRE, POST, DIff, and TERM I choose, so I think that signal distortion on the channel can be excluded, do you agree with me?
I have also noticed that links are easier to get locked with DFE Enabled = OFF on IBERT.
Is it possible that some CDR or resets related parameters have to be changed?
05-28-2018 06:55 AM
I too have noticed that the DFE sometimes makes locking worse, usually when the receive signal is strong. I'm thinking that you need to check for cross talk. Disable ALL the TX lanes except one, and then test that lane in isolation, to see how easily you can get it to lock. Then enable the adjacent lanes and test again.. If it won't lock, then crosstalk is an issue.. and not easily fixed.
I'm really puzzled by your statement that once the lanes are locked, you can vary parameters without causing errors... This does not sound right at all. Is this with a clock pattern, or with a PRBS pattern? If it's a clock pattern, it doesn't mean much.
05-28-2018 09:04 AM
I would start from the analysis of the channel, at least the description of the Insertion Loss. Then I would chose the RX EQ and TX strength/emphasis basing on this information. A further link optimization is always possible.
I would not change the CDR setup or the resets.
With IBERT you might scan the eye and measure the margin: let's try to achieve the highest margin with smallest TX swing. Please use a PRBS generator that is similar to the Aurora encoder (i.e. 8B10B -> PRBS7)
It is possible that the 80MHz REF clock is too low frequency. You should compare with the required phase noise masks.