GTH Transceiver data FIFO clock issue

[verilog_bee]

Here I simply post my query to the specific forum(serial transceiver) to get the proper response from the Xilinx Employee.

The original query is available at https://forums.xilinx.com/t5/UltraScale-Architecture/GTH-Transceiver-data-fifo-clock-issue/m-p/1015076/highlight/false#M11631

I am waiting for support on this issue, please help me out.

[verilog_bee]

Can you help me on this issue? @gguasti

[eschidl]

Hi @verilog_bee ,

in your block diagram it is not clear to me where your Clock A is coming from and where the TXUSRCLK is driven from. Where are the reference clocks of the TX and RX? Are they identical? Will the loopback always be there or do you plan to drive the RX with an asynchronous signal?

How do you achieve the clock correction? Do you use symbols for this? How often are they send?

[behnam_2705new]

you must consider that you have a 2 clock domain. you should consider that in your RD-FIFO you should read data from GT by rxuserclk2 and in your write FIFO you should use txuserclk2 to transmit. because you do not recover your clock the two rxuserclk2 and txuserclk2 are not same. if you want to pass your test. you should recover your clock with si570 or si5328 which exist in KCU105 board. I have done a similar project like this on KCU105. if you have a question, don't hesitate to contact me.

------------------------------------------------------------------------------
Don't forget to reply, give kudo and accept as solution

Thanks.
------------------------------------------------------------------------------

[verilog_bee]

in your block diagram, it is not clear to me where your Clock A is coming from and where the TXUSRCLK is driven from.

CLKA is design clock, all logic blocks of design working on that. Source for this clock is KCU105 onboard 125MHz clock(FPGAPin G10). To achieve the synchronous transfer of data to/from GTH we put Dual Clock FIFO at both sides between design & GTH transceiver.

TXUSRCLK is buffered output of TXOUTCLK output pin of GTH transceiver.

Where are the reference clocks of the TX and RX? Are they identical?

we are providing reference clock 125 Mhz to GTH (both tx & rx) at input port gtrefclk00_in.

Will the loopback always be there or do you plan to drive the RX with an asynchronous signal?

No, loopback is not permanent. The receiver source will be USB.

How do you achieve the clock correction? Do you use symbols for this? How often are they send?

At the moment there is no any clock correction symbol. But we will skip symbol for that moving forward.

[eschidl]

hi @verilog_bee ,

so to summarise, you are using quad 227 with the SI570 as refclk, set to 125MHz, and you use CLK_125MHZ from SI5335A. Do I see this right?

With that you are asynchronous on TX and RX in your FIFOs, and if you do not do any correction both FIFOs will over/underflow at some stage.

Why would you not use the reference clock for the transceivers as system clock? It has 125MHz too. And you would not need a FIFO on the TX side.

On the RX side, when you just use loopback, you would be synchronous too with the change mentioned above. But as you want to connect an external source you will be asynchronous and therefor need clock correction. Otherwise the FIFO will over/underflow, depending on the ppm difference of the clocks on both sides.

[verilog_bee]

so to summarise, you are using quad 227 with the SI570 as refclk, set to 125MHz,

This reference clock to GTH is an external clock comes from our FMC card. 125MHz.

We have tried our design also with onboard clock SI570 as reference clock(125MHz) & observed same results.

and you use CLK_125MHZ from SI5335A. Do I see this right?

Yes.

With that, you are asynchronous on TX and RX in your FIFOs, and if you do not do any correction both FIFOs will over/underflow at some stage.

Yes, That why we are using FIFO to synchronise data transfer between clocks.

Why would you not use the reference clock for the transceivers as system clock? It has 125MHz too. And you would not need a FIFO on the TX side.

We tried to use the output of .o of primitive IBUFDS_GTE3 as a design clock & reference clock but it fails in implementation level with the error. The error shows that we cannot use the reference clock to any other modules except GTH, that means it’s a dedicated clock for GTH.

please refer the post: 

Partially routed net error in GTH transceiver clock  in "serial transceiver" forum

Please guide us how can we use reference clock as a design clock.

On the RX side, when you just use loopback, you would be synchronous too with the change mentioned above. But as you want to connect an external source you will be asynchronous and therefor need clock correction. Otherwise the FIFO will over/underflow, depending on the ppm difference of the clocks on both sides.

Can you please guide us on required clock corrections.

[behnam_2705new]

you should make your RX and TX clock identical. probably your Rx clock has a deviation, you should use your SI5335 or Si570 to make the RX and TX clock similar If your RX clock has some PPMs your tx must has it too.

See 9.2. Synchronous Frequency Translation   part in SI5335 datasheet.

[andrewlan]

One way forward is by enabling clock correction symbols along with the elastic buffer. See UG576 starting from page 273 for information.

Also see:

https://forums.xilinx.com/t5/Serial-Transceivers/What-is-Elastic-buffer/td-p/810923

You will need to setup this operation on both ends of your link.

[eschidl]

Hi @verilog_bee ,

it seems you did not catch my meaning in the post you mentioned. You can use the transceiver reference clock inside fabric. You just did connect the wrong output port of the IBUFDS_GTE3 primitive for it.

The primitive has two outputs. The 'O' output can only be connected to the CHANNEL or COMMON primitive of the transceiver. But the 'ODIV2' output is for connections into fabric. And if you check with UG576, figure 2-1, you can see that you can select if you want to have the divide by 2 or not.

Regarding the clock correction, that would depend on your protocol. If you are using 8b10b encoding you could use the build in clock correction mechanism in the transceiver. Please have a look at UG576, page 268 and following for a description. If you use 64b66b encoding you might check if you can use the asynchronous gearbox or not. If you use another encoding you would have to implement this similarly yourself in fabric.

[verilog_bee]

Thanks for the quick response.

Let us verify our design with your suggested clock output of ODIV2.

We will inform you soon. 

[verilog_bee · Accepted solution]

Hello @eschidl 

I glad to know you that the issue with FIFO is resolved with your suggested clock scheme.

Suggested clock scheme:

1. External clock(125M) => IBUFDS_GTE3/O => GTH (125M)

                                  IBUFDS_GTE3/ODIV2 => BUFG_GT => Fabric logic(TX_FIFO_WR CLK & RX_FIFO_RD_CLK) (125M).

2.125M free running for DRP only.

Thanks to @eschidl.

Thanks for the support in my issue. @andrewlan @behnam_2705new  

Kudos!!!!!!!!